diff --git a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml index ffb1190f7e..95484d284b 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml @@ -21,8 +21,8 @@ en: 985e0eb63f3d1fd6 zh: 337cc44fadc6227e /agent-note-single-file-executable-sdk-runtime-distribution-single-exe/decision/build-pipeline-and-artifacts: - en: 13d4c28466577e97 - zh: 4fa5cf739d5727fe + en: 4a55602284b462e8 + zh: 762c4f8b719f179c /agent-note-single-file-executable-sdk-runtime-distribution-single-exe/decision/python-sdk-distribution-two-carriers-exe-for-production-node-for-development: en: b2a740a1ab7a5002 zh: a48be6ca6fc65912 diff --git a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md index 2f9c52b317..a70ab115be 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md +++ b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md @@ -48,7 +48,7 @@ The [Office kit](2026-09-14-independent-libreoffice-kit.md) and its installed de ### Build pipeline and artifacts -[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-python-runtime-closure deploy --prod --config.allow-unused-patches=true --config.node-linker=hoisted --config.pnpmfile=scripts/python-runtime-deploy.pnpmfile.mjs --config.link-workspace-packages=true --config.hoist-workspace-packages=false` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/`, including [`python/sdk-runtime/runtime-bootstrap.mjs`](../../../../python/sdk-runtime/runtime-bootstrap.mjs) as the carrier-root `runtime-bootstrap.mjs` → replace staged dependency symlinks with their target bytes, remove package-manager `.bin` links, and fail if any symlink remains → verify the deployed bootstrap and inject pkg configuration with that bin plus assets covering dynamic profile, bundle, frontend, preset, native-library, and configuration reads → stage the target `node-pty` addon → invoke `pkg --sea --compress Brotli` once per target, where the vendored pkg patch pins Brotli's quality to 6: pkg passes no options, so its default quality 11 measured 132.8 s against 3.2 s at quality 6 for an archive 12% larger → write `deepseek-harness-sdk-runtime--` under `dist-exe/` and copy it with its lightweight resource manifest and Office skill assets into the runtime directory. The Python runtime owns that bootstrap. It calls the public CLI export for ordinary launches and dispatches a provider-private selection to the same `@deepseek-ai/dsh-subprocess-local/runner` core without changing CLI grammar or adding another executable; the [native-containment reference](../../../../packages/subprocess/subprocess-local/README.md) owns that private path. Linux CI rebuilds `pty.node` inside the matching manylinux 2.28 container and copies the resulting addon into the deployed tree. Every target copies its native `@vscode/ripgrep` binary beside the executable as the required `-rg` sidecar; pkg runtimes select that sidecar through `process.pkg`, while ordinary Node execution uses `@vscode/ripgrep` directly. macOS uses its target prebuild and also emits the required `-spawn-helper`. The deploy flags preserve the measured runtime layout: the dedicated deploy lockfile retains the source lockfile’s versions and peer settings; the deploy-only pnpm hook translates reviewed local build approvals to exact file URLs without changing approval decisions; hoisted supplies package directories that staging makes symlink-free; staging removes nested workspace copies only when pnpm records the same dependency identity at the payload root, preserving singleton services; disabling workspace hoisting excludes unrelated workspace source packages; link-workspace-packages selects direct workspace dependencies. [`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) overrides the transitive `@deepseek-ai/cosmokit` and `@deepseek-ai/schemastery` semver requests to the pinned vendor sources so deploy never resolves those unpublished names from a registry. Production deployment permits unused workspace patches because development-only tools can be absent from its closure; included-package patch failures still fail deployment. Workspace installs retain strict unused-patch validation. +[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-python-runtime-closure deploy --prod --config.allow-unused-patches=true --config.node-linker=hoisted --config.pnpmfile=scripts/python-runtime-deploy.pnpmfile.mjs --config.link-workspace-packages=true --config.hoist-workspace-packages=false` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/`, including [`python/sdk-runtime/runtime-bootstrap.mjs`](../../../../python/sdk-runtime/runtime-bootstrap.mjs) as the carrier-root `runtime-bootstrap.mjs` → replace staged dependency symlinks with their target bytes, remove package-manager `.bin` links, and fail if any symlink remains → reject missing direct dependency directories, verify the deployed bootstrap, and inject pkg configuration with that bin plus assets covering dynamic profile, bundle, frontend, preset, native-library, and configuration reads → stage the target `node-pty` addon → invoke `pkg --sea --compress Brotli` once per target, where the vendored pkg patch pins Brotli's quality to 6: pkg passes no options, so its default quality 11 measured 132.8 s against 3.2 s at quality 6 for an archive 12% larger → write `deepseek-harness-sdk-runtime--` under `dist-exe/` and copy it with its lightweight resource manifest and Office skill assets into the runtime directory. The Python runtime owns that bootstrap. It calls the public CLI export for ordinary launches and dispatches a provider-private selection to the same `@deepseek-ai/dsh-subprocess-local/runner` core without changing CLI grammar or adding another executable; the [native-containment reference](../../../../packages/subprocess/subprocess-local/README.md) owns that private path. Linux CI rebuilds `pty.node` inside the matching manylinux 2.28 container and copies the resulting addon into the deployed tree. Every target copies its native `@vscode/ripgrep` binary beside the executable as the required `-rg` sidecar; pkg runtimes select that sidecar through `process.pkg`, while ordinary Node execution uses `@vscode/ripgrep` directly. macOS uses its target prebuild and also emits the required `-spawn-helper`. The deploy flags preserve the measured runtime layout: the dedicated deploy lockfile retains the source lockfile’s versions and peer settings; [`scripts/python-runtime-deploy.pnpmfile.mjs`](../../../../scripts/python-runtime-deploy.pnpmfile.mjs) preserves exact local build approvals; the [locked deployment decision](../bug-fix/2026-11-01-locked-python-runtime-deploy.md) supersedes the legacy deployment requirement and disabled-peer rationale; hoisted supplies package directories that staging makes symlink-free; staging removes nested workspace copies only when pnpm records the same dependency identity at the payload root, preserving singleton services; disabling workspace hoisting excludes unrelated workspace source packages; link-workspace-packages selects direct workspace dependencies. [`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) overrides the transitive `@deepseek-ai/cosmokit` and `@deepseek-ai/schemastery` semver requests to the pinned vendor sources so deploy never resolves those unpublished names from a registry. Production deployment permits unused workspace patches because development-only tools can be absent from its closure; included-package patch failures still fail deployment. Workspace installs retain strict unused-patch validation. CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml) runs [installed-wheel validation](../testing/2026-08-23-installed-python-wheel-black-box-ci.md) on Linux/Windows x64 for pull requests and Linux ARM64 plus both macOS architectures for master pushes. The [public publication workflow](../../archived/process/2026-08-11-python-publication-workflow.md) calls it for all five targets; `workflow_dispatch` can still select a subset. Native builds run on linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64 / macos-x64 (`macos-15-intel`) / win-x64 (`windows-2025`), with `~/.pkg-cache` cached where applicable, and pkg handles macOS ad-hoc signing. Each leg installs the release-shaped SDK and runtime wheels into a clean venv outside the checkout, verifies their package and executable origin, then drives the complete keyless scenario set through the public SDK and direct NDJSON JSON-RPC. Trusted pull requests and master pushes additionally run a real DeepSeek two-turn tool smoke on their selected targets; fork and Dependabot heads receive no key. Linux inspects the executable and native addon's GLIBC requirements and runs an additional manylinux 2.28 smoke, while macOS checks the runtime, ripgrep, and PTY helper architectures and verifies that all three deployment targets fit the wheel tag. A full five-target run retains six artifacts, each containing one release file: the platform-independent SDK wheel and five native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts `python-v` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and five native runtime wheels, then a single serialized job checks and publishes all six to the project PyPI registry. The [`python/sdk-runtime` README](../../../../python/sdk-runtime/README.md) owns the Windows target and the explicit exclusion of Windows arm64. diff --git a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md index ca1976c9bd..e1ff276504 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md @@ -48,7 +48,7 @@ exe 的 VFS 内是**构建产物形态的真实包树**(各包的 `lib/` + 真 ### 构建流水线与产物 -[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts):运行时闭包校验 → `pnpm run build` →(清空后)`pnpm --filter dsh-python-runtime-closure deploy --prod --config.allow-unused-patches=true --config.node-linker=hoisted --config.pnpmfile=scripts/python-runtime-deploy.pnpmfile.mjs --config.link-workspace-packages=true --config.hoist-workspace-packages=false` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/`,其中包含作为载体根目录 `runtime-bootstrap.mjs` 的 [`python/sdk-runtime/runtime-bootstrap.mjs`](../../../../python/sdk-runtime/runtime-bootstrap.mjs) → 将暂存依赖中的符号链接替换为目标文件内容,删除包管理器的 `.bin` 链接,并在仍有任何符号链接时失败 → 验证已部署的 bootstrap,并注入以该文件为 bin 的 pkg 配置及覆盖动态读取 profile、bundle、前端、preset、原生库与配置文件的 assets → 暂存目标平台的 `node-pty` addon → 每个构建目标调用一次 `pkg --sea --compress Brotli`,其中 vendor 的 pkg 补丁把 Brotli 质量固定为 6:pkg 不传参数时默认质量 11,同一载荷实测 132.8 s,质量 6 为 3.2 s,产物大 12% → 将 `deepseek-harness-sdk-runtime--` 写入 `dist-exe/`,并将它与轻量资源清单及 Office skill 资源一同拷回运行时目录。该 bootstrap 由 Python runtime 拥有;普通启动时它调用公开 CLI export,而提供方私有选择会分派到同一个 `@deepseek-ai/dsh-subprocess-local/runner` 核心,不改变 CLI 语法,也不增加另一个可执行文件;[原生 containment 参考](../../../../packages/subprocess/subprocess-local/README.zh.md)负责这条私有路径。Linux CI 会在匹配的 manylinux 2.28 容器中重新构建 `pty.node`,并将生成的 addon 复制到部署树中。每个目标都会把对应的原生 `@vscode/ripgrep` 二进制复制到可执行文件旁,作为必需的 `-rg` 伴随文件;pkg 运行时通过 `process.pkg` 选择该伴随文件,普通 Node 执行则直接使用 `@vscode/ripgrep`。macOS 使用对应目标的预构建产物,并额外生成所需的 `-spawn-helper`。部署标志都有实测依据:独立部署锁文件保留源锁文件的版本及对等依赖设置;仅用于部署的 pnpm hook 将已审阅的本地构建审批转换为精确文件 URL,不改变审批决定;`hoisted` 提供包目录,再由暂存步骤消除符号链接;只有 pnpm 在载荷根目录记录了相同依赖标识时,暂存步骤才删除对应的嵌套工作区副本,以保留服务单例;关闭工作区提升可排除无关的工作区源码包;`link-workspace-packages` 选择直接工作区依赖。[`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) 将传递的 `@deepseek-ai/cosmokit` 与 `@deepseek-ai/schemastery` semver 请求覆盖到固定的 vendor 源码,使 deploy 不会从注册表解析这些未发布名称。 生产部署允许未使用的工作区补丁,因为开发工具可能不属于运行时闭包;闭包内包的补丁应用失败仍会中止部署。工作区安装继续严格校验未使用的补丁。 +[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts):运行时闭包校验 → `pnpm run build` →(清空后)`pnpm --filter dsh-python-runtime-closure deploy --prod --config.allow-unused-patches=true --config.node-linker=hoisted --config.pnpmfile=scripts/python-runtime-deploy.pnpmfile.mjs --config.link-workspace-packages=true --config.hoist-workspace-packages=false` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/`,其中包含作为载体根目录 `runtime-bootstrap.mjs` 的 [`python/sdk-runtime/runtime-bootstrap.mjs`](../../../../python/sdk-runtime/runtime-bootstrap.mjs) → 将暂存依赖中的符号链接替换为目标文件内容,删除包管理器的 `.bin` 链接,并在仍有任何符号链接时失败 → 拒绝缺失的直接依赖目录,验证已部署的 bootstrap,并注入以该文件为 bin 的 pkg 配置及覆盖动态读取 profile、bundle、前端、preset、原生库与配置文件的 assets → 暂存目标平台的 `node-pty` addon → 每个构建目标调用一次 `pkg --sea --compress Brotli`,其中 vendor 的 pkg 补丁把 Brotli 质量固定为 6:pkg 不传参数时默认质量 11,同一载荷实测 132.8 s,质量 6 为 3.2 s,产物大 12% → 将 `deepseek-harness-sdk-runtime--` 写入 `dist-exe/`,并将它与轻量资源清单及 Office skill 资源一同拷回运行时目录。该 bootstrap 由 Python runtime 拥有;普通启动时它调用公开 CLI export,而提供方私有选择会分派到同一个 `@deepseek-ai/dsh-subprocess-local/runner` 核心,不改变 CLI 语法,也不增加另一个可执行文件;[原生 containment 参考](../../../../packages/subprocess/subprocess-local/README.zh.md)负责这条私有路径。Linux CI 会在匹配的 manylinux 2.28 容器中重新构建 `pty.node`,并将生成的 addon 复制到部署树中。每个目标都会把对应的原生 `@vscode/ripgrep` 二进制复制到可执行文件旁,作为必需的 `-rg` 伴随文件;pkg 运行时通过 `process.pkg` 选择该伴随文件,普通 Node 执行则直接使用 `@vscode/ripgrep`。macOS 使用对应目标的预构建产物,并额外生成所需的 `-spawn-helper`。部署标志都有实测依据:独立部署锁文件保留源锁文件的版本及对等依赖设置;[`scripts/python-runtime-deploy.pnpmfile.mjs`](../../../../scripts/python-runtime-deploy.pnpmfile.mjs) 保留精确的本地构建审批;[按锁文件部署的决策](../bug-fix/2026-11-01-locked-python-runtime-deploy.zh.md)替代 legacy 部署要求及关闭对等依赖的理由;`hoisted` 提供包目录,再由暂存步骤消除符号链接;只有 pnpm 在载荷根目录记录了相同依赖标识时,暂存步骤才删除对应的嵌套工作区副本,以保留服务单例;关闭工作区提升可排除无关的工作区源码包;`link-workspace-packages` 选择直接工作区依赖。[`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) 将传递的 `@deepseek-ai/cosmokit` 与 `@deepseek-ai/schemastery` semver 请求覆盖到固定的 vendor 源码,使 deploy 不会从注册表解析这些未发布名称。 生产部署允许未使用的工作区补丁,因为开发工具可能不属于运行时闭包;闭包内包的补丁应用失败仍会中止部署。工作区安装继续严格校验未使用的补丁。 CI 使用 [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml):[安装后 wheel 包验证](../testing/2026-08-23-installed-python-wheel-black-box-ci.zh.md)在拉取请求上运行 Linux/Windows x64,在 master 推送上运行 Linux ARM64 与两种 macOS 架构。[公开发布工作流](../../archived/process/2026-08-11-python-publication-workflow.md)调用它构建全部五个目标;`workflow_dispatch` 仍可选择部分目标。linux-x64、linux-arm64(`ubuntu-24.04-arm`)、macos-arm64、macos-x64(`macos-15-intel`)与 win-x64(`windows-2025`)分别进行原生构建,并在适用平台缓存 `~/.pkg-cache`;macOS 的 ad-hoc 签名由 pkg 处理。每个平台都把发布形态的 SDK wheel 包与运行时 wheel 包安装到 checkout 外的干净 venv,证明包与可执行文件来源,再通过公开 SDK 与直接 NDJSON JSON-RPC 运行完整 keyless 场景。可信拉取请求与 master 推送还会在各自选定的目标上运行真实 DeepSeek 双轮工具冒烟测试;fork 与 Dependabot head 不会获得密钥。Linux 会检查可执行文件和原生 addon 各自的 GLIBC 依赖,并额外运行 manylinux 2.28 冒烟测试;macOS 则检查 runtime、ripgrep 与 PTY helper 的架构,并验证三个载荷的部署目标都符合 wheel 包标签。完整构建五个目标时保留 6 个产物,每个产物只含一个发布文件:平台无关的 SDK wheel 包与 5 个原生运行时 wheel 包;手动选择部分目标时保留 SDK wheel 与所选运行时 wheel。裸 exe 与源码包只作为测试中间输入。[`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) 只接受版本与根目录 `package.json` 匹配的 `python-v` 标签流水线,构建一个 SDK wheel 包和 5 个原生运行时 wheel 包,再由单个串行任务校验并将这 6 个文件发布到项目的 PyPI 注册表。[`python/sdk-runtime` README](../../../../python/sdk-runtime/README.zh.md)负责 Windows 目标及对 Windows arm64 的明确排除。 diff --git a/.agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.i18n.yaml new file mode 100644 index 0000000000..618dcc06e2 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.i18n.yaml @@ -0,0 +1,22 @@ +# Bilingual-pair consistency record for 2026-11-01-locked-python-runtime-deploy.md (docs/i18n/README.md): per heading +# section, a hash of its English and Chinese blocks outside code blocks and generated regions. +# After editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.md +/agent-note-locked-python-runtime-deployment: + en: 6a06f1078957df51 + zh: d7c2146381443ced +/agent-note-locked-python-runtime-deployment/problem: + en: 2293006c8bae6a1a + zh: 10670f2d703777dd +/agent-note-locked-python-runtime-deployment/decision: + en: 223079c8e1e3a44e + zh: c906266cb0b231bc +/agent-note-locked-python-runtime-deployment/alternatives-considered: + en: 978c94c7013056a5 + zh: afae764120582b6a +/agent-note-locked-python-runtime-deployment/testing: + en: d862dd609725c686 + zh: 8889c5d9002bbc68 +/agent-note-locked-python-runtime-deployment/consequences: + en: 41827db476b30a43 + zh: 5a7652f310a69909 diff --git a/.agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.md b/.agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.md new file mode 100644 index 0000000000..a1763148c8 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.md @@ -0,0 +1,33 @@ +# Agent Note: Locked Python runtime deployment + +Status: implemented + +English | [中文](2026-11-01-locked-python-runtime-deploy.zh.md) + +## Problem + +Python executable packaging needs the same dependency versions and integrity records as the workspace lockfile. Legacy hoisted deployment ignores that lockfile and resolves compatible registry versions again: a manifest permitting `tslib@^2.8.1` can install `2.9.1` while the lockfile records `2.8.1`. Office archive preparation then rejects the missing locked integrity. A successful frozen workspace install does not constrain that later legacy deployment. + +## Decision + +The Python builder uses pnpm's default deployment, which derives a dedicated production lockfile from the workspace lockfile and installs it frozen. This replaces only the deployment strategy in the [single-executable distribution decision](../architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md); its runtime architecture and carrier layout remain active. + +Deployment retains the source lockfile's peer settings, including `autoInstallPeers: true`. Frozen installation preserves already resolved versions and peer relationships instead of resolving new registry peers. Forcing `autoInstallPeers: false` conflicts with the derived lockfile settings and fails frozen installation. The resulting tree need not equal the legacy tree resolved with peers disabled; selected dependencies must come from the locked graph. `verify-runtime-closure.ts` independently requires explicit workspace peers and preset plugins before deployment, and `verifyStagedDependencies` rejects missing direct dependency directories after materialization and deduplication. + +[`scripts/python-runtime-deploy.pnpmfile.mjs`](../../../../scripts/python-runtime-deploy.pnpmfile.mjs) translates reviewed local `file:` approvals to the exact absolute file URLs used in the derived lockfile. Approval, denial, pending decisions, and strict build validation remain unchanged. Default deployment executes approved installation scripts; their generated files can enter the staged payload. Workspace policy and the source lockfile remain unchanged. + +## Alternatives considered + +**Keep legacy deployment and add missing integrity records.** This would accept versions the source lockfile did not select and repeat the failure after another compatible release. + +**Disable automatic peer installation during default deployment.** This conflicts with the source lockfile settings. Re-resolving a separate graph would give up the source lockfile's authority. + +**Disable deployment scripts or approve local packages by name.** Skipping scripts omits required postinstall preparation. Name-only approvals do not authorize non-semver local file dependencies; exact identity translation preserves the existing review decisions. + +## Testing + +A real pnpm comparison with the same manifest and lockfile installs `tslib@2.9.1` through legacy deployment and retains `2.8.1` through default deployment. The Linux executable and locally built installed-wheel SDK smokes pass. Focused tests reject missing dependency directories and deployment flag regressions, preserve approval states, and exercise both metadata path bases. Cross-platform packaging and installed-wheel validation remain owned by CI. + +## Consequences + +Runtime versions remain traceable to the workspace lockfile, and incomplete direct dependency trees fail before executable packaging. Installation script outputs and locked peer selections can differ from legacy deployment, so byte-identical payloads are not promised. SSH helper deployment has a separate closure manifest and native payload validation; its legacy deployment is outside this Python runtime repair. diff --git a/.agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.zh.md b/.agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.zh.md new file mode 100644 index 0000000000..8b24c383e7 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-11-01-locked-python-runtime-deploy.zh.md @@ -0,0 +1,33 @@ +# Agent Note: Python 运行时按锁文件部署 + +Status: implemented + +[English](2026-11-01-locked-python-runtime-deploy.md) | 中文 + +## 问题 + +Python 可执行文件打包需要使用与工作区锁文件相同的依赖版本和完整性记录。legacy hoisted 部署忽略该锁文件,重新解析兼容的注册表版本:允许 `tslib@^2.8.1` 的 manifest 可能安装 `2.9.1`,而锁文件记录的是 `2.8.1`。Office 归档准备随后因缺少锁定的完整性记录而拒绝构建。工作区的冻结安装成功,并不能约束后续的 legacy 部署。 + +## 决策 + +Python 构建器使用 pnpm 的默认部署,从工作区锁文件派生独立的生产锁文件,再冻结安装。本记录只替代[单文件可执行程序分发决策](../architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md)中的部署策略;其中的运行时架构和载体布局继续有效。 + +部署保留源锁文件的对等依赖设置,包括 `autoInstallPeers: true`。冻结安装保留已解析的版本和对等依赖关系,不会重新解析新的注册表对等依赖。强制指定 `autoInstallPeers: false` 与派生锁文件的设置冲突,会使冻结安装失败。生成的依赖树不必与关闭对等依赖解析的 legacy 依赖树相同,但所选依赖必须来自锁定的依赖图。`verify-runtime-closure.ts` 独立要求在部署前显式声明工作区对等依赖及 preset 插件;`verifyStagedDependencies` 在链接实体化与去重后拒绝缺失的直接依赖目录。 + +[`scripts/python-runtime-deploy.pnpmfile.mjs`](../../../../scripts/python-runtime-deploy.pnpmfile.mjs) 将已审阅的本地 `file:` 审批转换为派生锁文件使用的精确绝对文件 URL。批准、拒绝、未决定状态和严格构建校验均保持不变。默认部署执行已批准的安装脚本,其生成的文件可能进入暂存载荷。工作区策略和源锁文件保持不变。 + +## 考虑过的替代方案 + +**保留 legacy 部署并补全缺失的完整性记录。** 这会接受源锁文件没有选择的版本,并在另一个兼容版本发布后重复出现故障。 + +**在默认部署中关闭对等依赖自动安装。** 这与源锁文件的设置冲突;重新解析另一套依赖图会放弃源锁文件的决定权。 + +**禁用部署脚本,或按名称批准本地包。** 跳过脚本会遗漏必需的 postinstall 准备。仅按名称批准不能授权非 semver 的本地文件依赖;转换精确标识可以保留原有审批决定。 + +## 测试 + +真实 pnpm 对照使用相同 manifest 和锁文件,legacy 部署安装 `tslib@2.9.1`,默认部署保留 `2.8.1`。Linux 可执行文件及本地构建并安装的 wheel SDK 冒烟测试通过。定向测试拒绝缺失的依赖目录和部署参数回退,保留审批状态,并覆盖两种 metadata 路径基准。跨平台打包及安装后 wheel 校验仍由 CI 负责。 + +## 影响 + +运行时版本可以追溯到工作区锁文件,不完整的直接依赖树会在可执行文件打包前失败。安装脚本产物和锁定的对等依赖选择可能与 legacy 部署不同,因此不承诺载荷逐字节相同。SSH helper 部署使用独立的闭包 manifest 和原生载荷校验,其 legacy 部署不属于本次 Python 运行时修复范围。 diff --git a/scripts/build-exe-for-python-sdk-staging.ts b/scripts/build-exe-for-python-sdk-staging.ts index 123ba52c33..4f766307f9 100644 --- a/scripts/build-exe-for-python-sdk-staging.ts +++ b/scripts/build-exe-for-python-sdk-staging.ts @@ -14,9 +14,9 @@ function isHoistedLocations(value: unknown): value is Record { * Different peer resolutions and third-party packages keep their installed locations. * All metadata paths are checked before deletion; the caller first materializes links. * @param staging - Symlink-free deployed payload root. - * @param workspace - Workspace root relative to which pnpm records hoisted locations. + * @param metadataRoot - Base for pnpm hoistedLocations: the workspace for legacy deploy, the deployment for default deploy. */ -export async function deduplicateStagedWorkspacePackages(staging: string, workspace: string): Promise { +export async function deduplicateStagedWorkspacePackages(staging: string, metadataRoot: string): Promise { const modules = resolve(staging, 'node_modules') const file = join(modules, '.modules.yaml') const manifest: unknown = load(await readFile(file, 'utf8')) @@ -27,7 +27,7 @@ export async function deduplicateStagedWorkspacePackages(staging: string, worksp const duplicates: string[] = [] for (const [identity, locations] of Object.entries(manifest.hoistedLocations)) { const paths = locations.map((location) => { - const path = resolve(workspace, location) + const path = resolve(metadataRoot, location) const local = relative(modules, path) if (local === '' || local === '..' || local.startsWith(`..${sep}`) || isAbsolute(local)) { throw new Error(`${file}: hoistedLocations path is outside deployed node_modules: ${location}`) @@ -62,7 +62,7 @@ async function findSymlink(directory: string): Promise { /** * Replace every deploy-time package link below the staged payload with real files. * - * Legacy deploy links workspace packages into the target's `node_modules`, and + * pnpm deploy can link workspace packages into the target's `node_modules`, and * those links must survive packaging as files. One link is not a package: when * the deploy target sits inside the deployed project, pnpm links that project * into its own `node_modules`. Dereferencing it would copy the payload into diff --git a/scripts/build-exe-for-python-sdk.spec.ts b/scripts/build-exe-for-python-sdk.spec.ts index fdaf7202a7..0d26340944 100644 --- a/scripts/build-exe-for-python-sdk.spec.ts +++ b/scripts/build-exe-for-python-sdk.spec.ts @@ -136,6 +136,9 @@ describe('Python runtime executable builder CLI', () => { const deploy = result.stdout.split('\n').find(line => line.includes(' --filter dsh-python-runtime-closure deploy')) expect(deploy).toContain('--prod --config.allow-unused-patches=true') expect(deploy).toContain('--config.hoist-workspace-packages=false') + expect(deploy).toContain(`--config.pnpmfile=${join(root, 'scripts/python-runtime-deploy.pnpmfile.mjs')}`) + expect(deploy).not.toContain('--legacy') + expect(deploy).not.toContain('--config.auto-install-peers=false') expect(result.stdout.split('--config.allow-unused-patches=true')).toHaveLength(2) expect(result.stdout).not.toContain(resolve(root, 'python/sdk-runtime/runtime-bootstrap.mjs')) expect(result.stdout).toContain('"bin":"runtime-bootstrap.mjs"') diff --git a/scripts/build-exe-for-python-sdk.ts b/scripts/build-exe-for-python-sdk.ts index 627c13f0e2..0d7b5d5624 100644 --- a/scripts/build-exe-for-python-sdk.ts +++ b/scripts/build-exe-for-python-sdk.ts @@ -9,7 +9,7 @@ import { createHash } from 'node:crypto' import { build as bundle } from 'tsdown' import { spawn } from 'node:child_process' -import { pnpmInvocation } from './executable-packaging.ts' +import { pnpmInvocation, verifyStagedDependencies } from './executable-packaging.ts' import { existsSync, statSync } from 'node:fs' import { chmod, copyFile, cp, glob, mkdir, readFile, readdir, rm, writeFile } from 'node:fs/promises' import { basename, dirname, join, resolve, sep } from 'node:path' @@ -334,6 +334,7 @@ class SingleExeBuild { } await materializeStagedLinks(this.staging) await deduplicateStagedWorkspacePackages(this.staging, this.staging) + await verifyStagedDependencies(this.staging) } /** Add the executable entry and pkg assets to the staged manifest. */ diff --git a/scripts/executable-packaging.spec.ts b/scripts/executable-packaging.spec.ts index 2adcbaaa89..35495f7907 100644 --- a/scripts/executable-packaging.spec.ts +++ b/scripts/executable-packaging.spec.ts @@ -2,7 +2,7 @@ import { lstat, mkdir, mkdtemp, readFile, rm, symlink, writeFile } from 'node:fs import { tmpdir } from 'node:os' import { join } from 'node:path' import { expect, it, onTestFinished } from 'vitest' -import { restoreLegacyHoists } from './executable-packaging.ts' +import { restoreLegacyHoists, verifyStagedDependencies } from './executable-packaging.ts' import { materializeStagedLinks } from './build-exe-for-python-sdk-staging.ts' async function fixture() { @@ -45,3 +45,20 @@ it('materializes a package junction without deleting its source and drops linked expect(await readFile(join(test.source, 'peer/index.js'), 'utf8')).toBe('intact') await expect(lstat(join(test.modules, '.bin'))).rejects.toThrow('ENOENT') }) + +it('accepts complete direct dependency directories without requiring optional dependencies', async () => { + const test = await fixture() + await writeFile(join(test.staging, 'package.json'), JSON.stringify({ + dependencies: { '@scope/present': '1.0.0' }, optionalDependencies: { optional: '1.0.0' }, + })) + await mkdir(join(test.modules, '@scope/present'), { recursive: true }) + await expect(verifyStagedDependencies(test.staging)).resolves.toBeUndefined() +}) + +it.each(['absent', 'file'])('rejects a direct dependency that is %s instead of a directory', async (kind) => { + const test = await fixture() + await writeFile(join(test.staging, 'package.json'), JSON.stringify({ dependencies: { missing: '1.0.0' } })) + if (kind === 'file') await writeFile(join(test.modules, 'missing'), '') + await expect(verifyStagedDependencies(test.staging)) + .rejects.toThrow('staged dependencies remain missing: missing.') +}) diff --git a/scripts/executable-packaging.ts b/scripts/executable-packaging.ts index 10248962bd..0643be28d6 100644 --- a/scripts/executable-packaging.ts +++ b/scripts/executable-packaging.ts @@ -1,5 +1,5 @@ /** Shared production-deploy preparation for the SDK and SSH executable carriers. */ -import { existsSync } from 'node:fs' +import { existsSync, statSync } from 'node:fs' import { cp, mkdir, readFile } from 'node:fs/promises' import { dirname, extname, join, resolve, sep } from 'node:path' @@ -61,10 +61,22 @@ export async function restoreLegacyHoists(staging: string, sourceNodeModules: st }) restored.push(dependency) } - const stillMissing = Object.keys(manifest.dependencies ?? {}) - .filter(dependency => !existsSync(join(staging, 'node_modules', dependency))) - if (stillMissing.length > 0) { - throw new Error(`executable-packaging: staged dependencies remain missing: ${stillMissing.join(', ')}.`) - } + await verifyStagedDependencies(staging) return restored } + +/** + * Reject a deployed payload whose manifest names missing direct dependency directories. + * @param staging - Deployed payload root containing package.json and node_modules. + * @returns Resolves when every declared direct dependency has a directory. + */ +export async function verifyStagedDependencies(staging: string): Promise { + const manifest = JSON.parse(await readFile(join(staging, 'package.json'), 'utf8')) as { + dependencies?: Record + } + const missing = Object.keys(manifest.dependencies ?? {}).sort() + .filter(dependency => !statSync(join(staging, 'node_modules', dependency), { throwIfNoEntry: false })?.isDirectory()) + if (missing.length > 0) { + throw new Error(`executable-packaging: staged dependencies remain missing: ${missing.join(', ')}.`) + } +} diff --git a/scripts/python-runtime-deploy.pnpmfile.mjs b/scripts/python-runtime-deploy.pnpmfile.mjs index d3cd00ff71..bf2c4fa1ee 100644 --- a/scripts/python-runtime-deploy.pnpmfile.mjs +++ b/scripts/python-runtime-deploy.pnpmfile.mjs @@ -1,11 +1,14 @@ -/** Preserve workspace build approvals when pnpm deploy converts local dependencies to file URLs. */ +/** + * Preserve workspace build approvals when pnpm deploy records local dependencies as file:// URLs. + * This hook lives in the workspace root's scripts/ directory; relative file: approvals use that root. + */ import { resolve } from 'node:path' import { pathToFileURL } from 'node:url' /** * Translate local build identities without changing their approved or denied status. - * @param {Record & { allowBuilds?: Record }} config - Source workspace configuration. - * @returns {Record} Deployment configuration with exact file URL approvals. + * @param {import('./python-runtime-deploy.pnpmfile.mjs').RuntimeDeployConfig} config - Source workspace configuration. + * @returns {import('./python-runtime-deploy.pnpmfile.mjs').RuntimeDeployConfig} Deployment configuration with exact file URL approvals. */ function updateConfig(config) { if (config.allowBuilds === undefined) return config diff --git a/scripts/verify-runtime-closure.ts b/scripts/verify-runtime-closure.ts index 8694dcaa7d..84a4edbe72 100644 --- a/scripts/verify-runtime-closure.ts +++ b/scripts/verify-runtime-closure.ts @@ -1,8 +1,8 @@ /** * Verify that the executable deploy manifest supplies every plugin referenced * by a shipped agent preset and every required workspace peer in its dependency - * graph. With auto peer installation disabled, either omission can otherwise - * fail only when Cordis loads the packaged plugin. + * graph. Frozen deployment preserves the locked peer graph; this check also + * requires explicit workspace peers so omissions fail before plugin loading. */ import { globSync } from 'node:fs' import { readFile } from 'node:fs/promises'