Merge pull request #8014 from deepseek-harness/luanlai/ptc-embedded-persistent
feat(ptc): 支持 embedded 模式启用 persistent PTC
This commit is contained in:
commit
b1fa53b978
9 files changed
+62
-23
No files matched your search
@@ -24,8 +24,8 @@
|
||||
en: 2a9fa3292bf07355
|
||||
zh: db0b5684820915eb
|
||||
/deepseek-ai-dsh-ptc-runtime-node/use-this-package/in-memory-modes:
|
||||
en: c82e89a3221bfa3f
|
||||
zh: 5b37a48d50291a06
|
||||
en: 41c54a052af70659
|
||||
zh: becb41300547089e
|
||||
/deepseek-ai-dsh-ptc-runtime-node/use-this-package/deadlines-and-cancellation:
|
||||
en: d383ec7aeab8afec
|
||||
zh: 7c9c49e205fbbcd8
|
||||
@@ -39,8 +39,8 @@
|
||||
en: f27d477e3ccb10de
|
||||
zh: 0f14a4899bae6c39
|
||||
/deepseek-ai-dsh-ptc-runtime-node/understand-the-implementation/source-and-built-bootstraps:
|
||||
en: 93c117a1e404ddca
|
||||
zh: 6902de0d96efb03b
|
||||
en: b944263ce0b1d200
|
||||
zh: c25a99f01a0d9af5
|
||||
/deepseek-ai-dsh-ptc-runtime-node/understand-the-implementation/runner-import-handling:
|
||||
en: e2458b4653a620bb
|
||||
zh: 4ae26bd1e6a69430
|
||||
|
||||
@@ -91,7 +91,7 @@ The `persistent-in-memory` mode and Script mode with `scriptPersistence: true` r
|
||||
executionMode: persistent-in-memory
|
||||
```
|
||||
|
||||
Save the patch and pass its path to `dsh headless --patch <path>` in a composition with PTC tool presentation. The `per-call` mode and Script mode with `scriptPersistence: false` retain no program memory between calls. Changing the configuration reloads the provider and releases its execution resources. Packaged single-executable hosts require a `node-script` launch with `launch.executable` pointing to Node and `launch.bootstrapPath` pointing to this package's built `process.js` for in-memory execution.
|
||||
Save the patch and pass its path to `dsh headless --patch <path>` in a composition with PTC tool presentation. The `per-call` mode and Script mode with `scriptPersistence: false` retain no program memory between calls. Changing the configuration reloads the provider and releases its execution resources. Packaged single-executable hosts use their embedded carrier by default; the host sends the selected program format and retention flag with each evaluation, so the embedded `process.js` bootstrap serves both per-call and retained execution. Remote script deployments use a `node-script` launch with `launch.bootstrapPath` pointing to this package's built `persistent-process.js`.
|
||||
|
||||
In REPL mode, each live Agent keeps its own process and top-level variables, functions, and imported modules across `run_code` calls and turns. Use `await import(...)`, `console.log(...)`, and a final JSON expression; top-level `return` is not supported. Assignments to undeclared names create retained globals under Node REPL’s non-strict evaluation. `_` and `_error` are ordinary retained globals, without interactive REPL result setters. Node REPL declaration rules apply, including errors on redeclaring existing `let`, `const`, or `class` names; assign to a mutable binding or choose a new name. A leading `{ ... }` is a statement block and can contain top-level await and temporary declarations; parenthesize an object expression to return it. Retained functions and saved tool methods call the current evaluation's bindings. All tool calls must be awaited; callbacks from a completed evaluation cannot invoke tools in a later evaluation.
|
||||
|
||||
@@ -137,7 +137,7 @@ Per-call completion leaves unfinished binding waits pending and stops binding ca
|
||||
|
||||
### Source and built bootstraps
|
||||
|
||||
Source execution loads an erasable-only bootstrap closure without relying on sibling built exports; its evaluator modules and installed Acorn and birpc dependencies must be available in the execution world. Built script execution uses `process.js` for every mode; `persistent-process.js` delegates to the same loop with a REPL default for hosts that omit program syntax. Persistent mode requires a `node-script` launch. A remote script deployment supplies the matching `launch.bootstrapPath`. Embedded carriers launch their private worker through `launch.executable`. `resolveLaunch()` selects the local default during construction; an explicit remote invocation never depends on whether the client itself is packaged. A host path is never assumed to name the same remote file.
|
||||
Source execution loads an erasable-only bootstrap closure without relying on sibling built exports; its evaluator modules and installed Acorn and birpc dependencies must be available in the execution world. Built script execution uses `process.js` for every mode; `persistent-process.js` delegates to the same loop with a REPL default for hosts that omit program syntax. Embedded carriers launch their private worker through `launch.executable` and carry the program format and retention flag in each evaluation. A remote script deployment supplies the matching `launch.bootstrapPath`. `resolveLaunch()` selects the local default during construction; an explicit remote invocation never depends on whether the client itself is packaged. A host path is never assumed to name the same remote file.
|
||||
|
||||
Source containing `await` is checked for top-level await before native compilation, which can otherwise treat `await` as an identifier. Acorn parses an async wrapper, and acorn-walk traverses it using Node's REPL visitors, declaration hoisting and character-position edits. Computed class-method keys are traversed while method bodies remain separate evaluations. The transform boxes the final expression; the evaluator unwraps and awaits its value. Source that needs no lowering uses native compilation; submissions without `await` retain native syntax diagnostics. Syntax errors preserve the parser message and available source location and caret, without compiler or transformer stack frames. Dynamic `import()` expressions remain native and use the compiling Script's `importModuleDynamically` option. Both private process bundles include Acorn, acorn-walk and birpc; source execution imports the installed packages.
|
||||
|
||||
|
||||
@@ -91,7 +91,7 @@ profile 编辑器可从 `@deepseek-ai/dsh-ptc-runtime-node/configuration` 导入
|
||||
executionMode: persistent-in-memory
|
||||
```
|
||||
|
||||
保存补丁,在使用 PTC 工具展示的配置中通过 `dsh headless --patch <path>` 加载。`per-call` 模式和 `scriptPersistence: false` 的 Script 模式不跨调用保留程序内存。修改配置会重载提供方并释放其执行资源。单文件打包宿主使用内存保留模式时,需要使用 `node-script` 启动方式,将 `launch.executable` 指向 Node,并将 `launch.bootstrapPath` 指向本包构建出的 `process.js`。
|
||||
保存补丁,在使用 PTC 工具展示的配置中通过 `dsh headless --patch <path>` 加载。`per-call` 模式和 `scriptPersistence: false` 的 Script 模式不跨调用保留程序内存。修改配置会重载提供方并释放其执行资源。单文件打包宿主默认使用内嵌载体;宿主在每次执行中发送所选的程序格式和保留标志,因此内嵌的 `process.js` bootstrap 同时服务单次执行和保留执行。远端脚本部署使用 `node-script` 启动方式,并将 `launch.bootstrapPath` 指向本包构建出的 `persistent-process.js`。
|
||||
|
||||
在 REPL 模式下,每个存活 Agent 保留独立进程,跨 `run_code` 调用和轮次保留顶层变量、函数及导入模块。使用 `await import(...)`、`console.log(...)` 和末尾 JSON 表达式;不支持顶层 `return`。Node REPL 使用非严格模式,给未声明的名称赋值会创建保留的全局变量。`_` 和 `_error` 是普通的保留全局变量,不使用交互式 REPL 的结果设置器。声明遵循 Node REPL 规则,包括重复声明已有 `let`、`const` 或 `class` 名称时报错;应给可变绑定赋值或选择新名称。开头的 `{ ... }` 按语句块执行,可以包含顶层 await 和临时声明;需要返回对象表达式时,用圆括号包裹。保留的函数和工具方法引用调用当前执行的绑定。必须等待所有工具调用;已完成执行的回调不能在后续执行中调用工具。
|
||||
|
||||
@@ -137,7 +137,7 @@ Host 通过有序控制管道上的 birpc 调用子进程的 `run()`。同一时
|
||||
|
||||
### 源代码与构建后 bootstrap
|
||||
|
||||
源代码执行加载仅含可擦除语法的 bootstrap 依赖,不依赖同级包的构建后导出;执行世界中必须提供求值器模块和已安装的 Acorn 与 birpc 依赖。构建后脚本在所有模式下都使用 `process.js`;`persistent-process.js` 委托同一个循环,供未指定程序语法的宿主默认使用 REPL。保留模式要求 `node-script` 启动方式。远端脚本部署提供对应的 `launch.bootstrapPath`。内嵌载体通过 `launch.executable` 启动私有 worker。`resolveLaunch()` 在构造时选择本地默认值;显式的远端调用不依赖客户端自身是否打包。不会假设 Host 路径对应同一个远程文件。
|
||||
源代码执行加载仅含可擦除语法的 bootstrap 依赖,不依赖同级包的构建后导出;执行世界中必须提供求值器模块和已安装的 Acorn 与 birpc 依赖。构建后脚本对所有模式使用 `process.js`;`persistent-process.js` 委托同一个循环,并为未指定程序语法的宿主提供 REPL 默认值。内嵌载体通过 `launch.executable` 启动私有 worker,并在每次执行中携带程序格式和保留标志。远端脚本部署提供对应的 `launch.bootstrapPath`。`resolveLaunch()` 在构造时选择本地默认值;显式的远端调用不依赖客户端自身是否打包。不会假设 Host 路径对应同一个远程文件。
|
||||
|
||||
包含 `await` 的源码会在原生编译前检查顶层 await,否则原生编译可能将 `await` 当作标识符。Acorn 解析包住源码的异步函数,acorn-walk 沿用 Node REPL 的 visitor、声明提升和按字符位置修改源码的方式进行遍历。类方法的计算属性名参与遍历,方法体仍独立求值。转换包装末尾表达式,求值器解包并等待其值。无需转换的源码使用原生编译;不含 `await` 的提交保留原生语法诊断。语法错误保留解析器消息、可用的源码位置与光标,不包含编译器或转换器的调用帧。动态 `import()` 表达式保持原生形式,使用编译该 Script 时的 `importModuleDynamically` 选项。两个私有进程 bundle 都包含 Acorn、acorn-walk 和 birpc;源码执行导入已安装的包。
|
||||
|
||||
|
||||
@@ -52,9 +52,7 @@ class NodeExecution {
|
||||
constructor(
|
||||
private readonly ctx: Context, private readonly config: ResolvedConfig,
|
||||
private readonly processes: PolicyProcesses, readonly mode: ExecutionMode,
|
||||
) {
|
||||
if (mode.retainsState && this.config.launch.kind === 'embedded') throw new Error('Persistent PTC requires a node-script launch')
|
||||
}
|
||||
) {}
|
||||
|
||||
/** Stop every owned process, retaining failed cleanup for a later retry. */
|
||||
async dispose(): Promise<void> {
|
||||
|
||||
@@ -38,13 +38,11 @@ export function resolveLaunch(config?: LaunchConfig): LaunchConfig {
|
||||
* @param fs - Filesystem mapping host bootstrap assets into the process world.
|
||||
* @param config - resolved worker invocation.
|
||||
* @param maxMessageBytes - Validated frame and queued-write limit.
|
||||
* @param persistent - Whether the selected mode requires retained-process launch support.
|
||||
* @returns Arguments following the resolved Node executable.
|
||||
*/
|
||||
export function bootstrapArgs(fs: Pick<FileSystem, 'processPathFromHostPath'>, config: LaunchConfig, maxMessageBytes: number, persistent = false): string[] {
|
||||
export function bootstrapArgs(fs: Pick<FileSystem, 'processPathFromHostPath'>, config: LaunchConfig, maxMessageBytes: number): string[] {
|
||||
switch (config.kind) {
|
||||
case 'embedded':
|
||||
if (persistent) throw new Error('Persistent PTC requires a node-script launch')
|
||||
return [String(maxMessageBytes)]
|
||||
case 'node-script': return ['--experimental-vm-modules', ...scriptBootstrapArgs(fs, config.bootstrapPath, maxMessageBytes)]
|
||||
/* v8 ignore next -- Callers supply the resolved, closed LaunchConfig union. */
|
||||
|
||||
@@ -282,7 +282,7 @@ export class ProgramWorker {
|
||||
const packaged = this.config.launch.kind === 'embedded'
|
||||
const heapFlag = `--max-old-space-size=${this.config.maxOldGenerationSizeMb}`
|
||||
const argv = [executable, ...packaged ? [] : [heapFlag],
|
||||
...bootstrapArgs(this.ctx.fs, this.config.launch, this.config.maxMessageBytes, this.retainsState)]
|
||||
...bootstrapArgs(this.ctx.fs, this.config.launch, this.config.maxMessageBytes)]
|
||||
const policy = this.policy
|
||||
this.confined = policy.mode === 'danger-full-access' ? undefined : await this.ctx.sandbox.confine(argv, { ...policy, mode: policy.mode }, this.controller.signal)
|
||||
// oxlint-disable-next-line typescript/no-unnecessary-condition -- Confinement may yield while disposal closes the process.
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import { describe, expect, it, onTestFinished } from 'vitest'
|
||||
import { mountRuntime } from './setup.ts'
|
||||
|
||||
const runtimePath = process.env.DSH_PACKAGED_RUNTIME ?? ''
|
||||
|
||||
describe.skipIf(runtimePath === '')('packaged embedded carrier', () => {
|
||||
it('retains state across evaluations through the packaged runtime', async () => {
|
||||
const cwd = await mkdtemp(join(tmpdir(), 'dsh-packaged-ptc-'))
|
||||
const ctx = new Context()
|
||||
onTestFinished(async () => { await ctx.fiber.dispose(); await rm(cwd, { recursive: true, force: true }) })
|
||||
const { runtime } = await mountRuntime(ctx, {
|
||||
executionMode: 'persistent-in-memory',
|
||||
launch: { kind: 'embedded', executable: runtimePath },
|
||||
})
|
||||
const owner = {}
|
||||
const run = (program: string) =>
|
||||
runtime.run(runtime.resolve({ program, bindings: [], cwd, stateOwner: owner }))
|
||||
|
||||
expect((await run('var retained = 40; retained')).value).toBe(40)
|
||||
expect((await run('retained += 2; retained')).value).toBe(42)
|
||||
await runtime.release(owner)
|
||||
expect((await run('typeof retained')).value).toBe('undefined')
|
||||
}, 120_000)
|
||||
|
||||
it('does not retain state without an owner', async () => {
|
||||
const cwd = await mkdtemp(join(tmpdir(), 'dsh-packaged-ptc-'))
|
||||
const ctx = new Context()
|
||||
onTestFinished(async () => { await ctx.fiber.dispose(); await rm(cwd, { recursive: true, force: true }) })
|
||||
const { runtime } = await mountRuntime(ctx, {
|
||||
launch: { kind: 'embedded', executable: runtimePath },
|
||||
})
|
||||
const run = (program: string) =>
|
||||
runtime.run(runtime.resolve({ program, bindings: [], cwd }))
|
||||
|
||||
expect((await run('var retained = 40; return retained')).value).toBe(40)
|
||||
expect((await run('return typeof retained')).value).toBe('undefined')
|
||||
}, 120_000)
|
||||
})
|
||||
@@ -34,9 +34,9 @@ it('resolves the local carrier once while preserving an explicit remote script c
|
||||
expect(resolveLaunch()).toEqual({ kind: 'node-script', executable: process.execPath })
|
||||
})
|
||||
|
||||
it('requires a script launch for persistent processes and selects their bootstrap', () => {
|
||||
expect(() => bootstrapArgs(noMapping, { kind: 'embedded', executable: '/remote/helper' }, 2048, true))
|
||||
.toThrow('requires a node-script launch')
|
||||
expect(bootstrapArgs(noMapping, { kind: 'node-script', executable: '/remote/node', bootstrapPath: '/remote/persistent-process.js' }, 2048, true))
|
||||
it('selects embedded and script bootstrap arguments', () => {
|
||||
expect(bootstrapArgs(noMapping, { kind: 'embedded', executable: '/remote/helper' }, 2048))
|
||||
.toEqual(['2048'])
|
||||
expect(bootstrapArgs(noMapping, { kind: 'node-script', executable: '/remote/node', bootstrapPath: '/remote/persistent-process.js' }, 2048))
|
||||
.toEqual(['--experimental-vm-modules', '/remote/persistent-process.js', '2048'])
|
||||
})
|
||||
@@ -372,13 +372,14 @@ describe.each([
|
||||
expect((await run('typeof retained')).value).toBe('undefined')
|
||||
})
|
||||
|
||||
it('rejects a packaged host without a persistent bootstrap during provider loading', async () => {
|
||||
it('accepts an embedded carrier for a packaged host', async () => {
|
||||
const previous = Object.getOwnPropertyDescriptor(process, 'pkg')
|
||||
try {
|
||||
Object.defineProperty(process, 'pkg', { configurable: true, value: {} })
|
||||
await expect(setup()).rejects.toThrow('requires a node-script launch')
|
||||
const { runtime } = await setup({ launch: { kind: 'node-script', executable: process.execPath, bootstrapPath: join(process.cwd(), 'persistent-process.js') } })
|
||||
expect(runtime.programFormat).toBe(format)
|
||||
const embedded = await setup()
|
||||
expect(embedded.runtime.programFormat).toBe(format)
|
||||
const scripted = await setup({ launch: { kind: 'node-script', executable: process.execPath, bootstrapPath: join(process.cwd(), 'persistent-process.js') } })
|
||||
expect(scripted.runtime.programFormat).toBe(format)
|
||||
} finally {
|
||||
if (previous === undefined) Reflect.deleteProperty(process, 'pkg')
|
||||
else Object.defineProperty(process, 'pkg', previous)
|
||||
|
||||
Reference in new issue
Block a user