fix: validate optional Python resources and isolate downloads
This commit is contained in:
1 parent
305021fcd6
commit
4fd85be130
19 files changed
+335
-149
No files matched your search
@@ -6,14 +6,14 @@
|
||||
en: 1ea1d8e6d2c14b02
|
||||
zh: 8e0b9903158667a1
|
||||
/deepseek-harness-runtime-bin/installed-commands-and-artifacts:
|
||||
en: fb7f6c3cec95833b
|
||||
zh: b04e60cbc0ba96fc
|
||||
en: 89c97974ad5e97a6
|
||||
zh: af99634672c80994
|
||||
/deepseek-harness-runtime-bin/python-module-api:
|
||||
en: 78a200e328d96637
|
||||
zh: f3b7a0b69b7a39e0
|
||||
/deepseek-harness-runtime-bin/packaged-profile-resolution:
|
||||
en: 30b9d8f69da71b23
|
||||
zh: 9eb984af84dc7ead
|
||||
en: f1123b8835b57bbe
|
||||
zh: 05af612c33c5d855
|
||||
/deepseek-harness-runtime-bin/build-and-distribution:
|
||||
en: 1973f696bc202247
|
||||
zh: 4642f2414b1618f2
|
||||
en: 01b6d838deae9c7b
|
||||
zh: 88b09ab5f1ae14ca
|
||||
@@ -21,11 +21,11 @@ office = download_office()
|
||||
primary_runtime = download_primary_runtime()
|
||||
```
|
||||
|
||||
`download_office()` downloads the locked npm Kit, target engine, and complete dependency tree, plus standalone Node for the Kit CLI. `download_primary_runtime()` independently downloads CPython, locked Python libraries, Node, pnpm, and Office skills. Both return absolute resource directories and reuse completed caches. The existing local `installPrimaryRuntime()` operation still copies an already available payload; it does not download.
|
||||
`download_office()` downloads the locked npm Kit, target engine, and complete dependency tree, plus standalone Node for the Kit CLI. `download_primary_runtime()` independently downloads CPython, locked Python libraries, Node, pnpm, and Office skills. Both return absolute resource directories and reuse completed caches. The authoring download validates its target, interpreters, library directory and skills before returning. The existing local `installPrimaryRuntime()` operation still copies an already available payload; it does not download.
|
||||
|
||||
`DSH_RESOURCE_CACHE` selects an absolute cache root, defaulting to `~/.cache/deepseek-harness/resources`. Resource identities include the release's locked inputs. Downloads verify archive hashes and package identities, preserve executable permissions, and publish completed directories atomically. Failed downloads can be retried. `DSH_RESOURCE_DOWNLOAD_TIMEOUT_MS` bounds each archive download, including its body; the default is `300000`, and accepted values are integers from `1` to `2147483647`. Downloads follow the Harness proxy environment policy. Installed wheel files are never modified.
|
||||
`DSH_RESOURCE_CACHE` selects an absolute cache root, defaulting to `~/.cache/deepseek-harness/resources`. Resource identities include the release's locked inputs. Downloads verify archive hashes and package identities, preserve executable permissions, and publish completed directories atomically. Failed downloads can be retried. `DSH_RESOURCE_DOWNLOAD_TIMEOUT_MS` bounds each archive download, including its body; the default is `300000`, and accepted values are integers from `1` to `2147483647`. Downloads follow the Harness proxy environment policy. Installed wheel files are never modified. Release versions have separate cache directories; unused versions require manual removal. Downloads use the locked upstream URLs without a registry or mirror override.
|
||||
|
||||
On subsequent launches the packaged bootstrap exposes downloaded resources to the SDK profile. `DSH_PRIMARY_RUNTIME` overrides the cached authoring environment; an empty value disables its query. `DSH_OFFICE_SIDECAR` can select an absolute sidecar directory with its standalone Node. Office skills are available when either resource is selected; authoring-only deployments expose the skills with the Kit CLI disabled. External authoring payloads retain the `primary-runtime/` plus sibling `office-skills/` layout. The SDK reads resources in place without copying them into `DSH_HOME`. The `sdk-minimal` profile does not mount these providers.
|
||||
On subsequent launches the packaged bootstrap exposes downloaded resources to the SDK profile. `DSH_PRIMARY_RUNTIME` overrides the cached authoring environment; an empty value disables its query. `DSH_OFFICE_SIDECAR` can select an absolute sidecar directory with its standalone Node. The Office checker is included in the wheel; users may provide their own Python environment and document libraries without downloading the authoring environment. Office skills are available when either resource is selected; authoring-only deployments expose the skills with the Kit CLI disabled. External authoring payloads retain the `primary-runtime/` plus sibling `office-skills/` layout. The SDK reads resources in place without copying them into `DSH_HOME`. The `sdk-minimal` profile does not mount these providers.
|
||||
|
||||
Skill selection is independent of delivery: project, custom-directory, and user filesystem skills override same-name bundled skills. An SDK patch can disable only the Office provider while retaining the Python query:
|
||||
|
||||
@@ -56,7 +56,7 @@ Unsupported platforms and missing executables or sidecars raise `FileNotFoundErr
|
||||
|
||||
`dsh` initializes shipped profiles under the explicit home, composes their bundle patches, and loads bundled plugins from the executable's virtual filesystem. Runtime resolution uses an in-memory generation instead of disk symlinks or proxy packages. Fallback imports use recorded declaring-package paths, including paths inside the executable's virtual filesystem, so built-in rows and external plugin peers share the bundled Cordis/module instance. Native shared libraries and Windows ConPTY addons are packaged with native addons, while ripgrep and the macOS PTY helper remain executable sidecars.
|
||||
|
||||
The Python bootstrap resolves the Office kit from its explicitly downloaded or selected sidecar so native helpers and URL Workers use real filesystem paths. The same resource preparation implementation serves repository builds and explicit Python downloads. The kit owns document operations; the Office smoke retrieves CLI paths from the loaded skill and executes capabilities and DOCX conversion with an empty PATH.
|
||||
Python and the packaged bootstrap share the resource layout: `<cache>/<identity>/office/`, `node/bin/node[.exe]`, and `node_modules/@deepseek-ai/libreoffice-kit/lib/cli.js`. Automatic Office cache discovery requires the `complete` marker to match the manifest identity; an explicit `DSH_OFFICE_SIDECAR` requires the Node and CLI files. The Python bootstrap resolves the Office kit from its explicitly downloaded or selected sidecar so native helpers and URL Workers use real filesystem paths. The same resource preparation implementation serves repository builds and explicit Python downloads. The kit owns document operations; the Office smoke retrieves CLI paths from the loaded skill and executes capabilities and DOCX conversion with an empty PATH.
|
||||
|
||||
External profile management uses `dsh plugin --profile <name> ...`. That command requires `pnpm` on `PATH`; ordinary SDK/profile execution does not.
|
||||
|
||||
@@ -64,6 +64,6 @@ External profile management uses `dsh plugin --profile <name> ...`. That command
|
||||
|
||||
Production deployment permits unused workspace patches for packages outside the runtime closure; patches for included packages must still apply successfully. This exception is confined to the deploy command; repository installation still rejects unused patches.
|
||||
|
||||
From the repository root, `pnpm exec tsx scripts/build-exe-for-python-sdk.ts` verifies the closure, builds packages, deploys a symlink-free tree, packages the selected target, and syncs the executable and sidecars into this module. Only the target platform’s PTY prebuilds are packaged. Package README, CHANGELOG, HISTORY Markdown, source maps, and TypeScript declarations outside runtime assets are omitted. Deployment disables workspace hoisting, and packaging excludes the checkout’s node_modules so dependency scans cannot add development packages to the payload. Staging shares root workspace packages with nested consumers of the same pnpm dependency identity, preserving singleton services while retaining different peer resolutions. `scripts/build-python-release.py` stages release-shaped wheels at the root repository version and pins `deepseek-harness-sdk` to the exact runtime version.
|
||||
From the repository root, `pnpm exec tsx scripts/build-exe-for-python-sdk.ts` verifies the closure, builds packages, deploys a symlink-free tree, packages the selected target, and syncs the executable and sidecars into this module. Only the target platform’s PTY prebuilds are packaged. Package README, CHANGELOG, HISTORY Markdown, source maps, and TypeScript declarations outside runtime assets are omitted. Deployment disables workspace hoisting, and packaging excludes checkout dependencies and workspace source trees outside the deployment so dependency scans cannot add development packages to the payload. Staging shares root workspace packages with nested consumers of the same pnpm dependency identity, preserving singleton services while retaining different peer resolutions. `scripts/build-python-release.py` stages release-shaped wheels at the root repository version and pins `deepseek-harness-sdk` to the exact runtime version.
|
||||
|
||||
The installed-wheel smoke creates a clean virtual environment outside the checkout, proves the installed distribution and executable identities, then exercises default and customized SDK profiles, external plugins, MCP, native tools, direct JSON-RPC, committed snapshots, and the real provider on trusted runs. Its Office scenario relocates the lightweight target payload, explicitly downloads the sidecar, and converts DOCX with the required platform engine: the target’s declared native engine, or WASM when no native engine is declared. See the [Python contributor workflow](../development.md) and [installed-wheel testing decision](../../.agents/notes/implemented/testing/2026-08-23-installed-python-wheel-black-box-ci.md).
|
||||
@@ -21,11 +21,11 @@ office = download_office()
|
||||
primary_runtime = download_primary_runtime()
|
||||
```
|
||||
|
||||
`download_office()` 下载锁定版本的 npm Kit、目标引擎及完整依赖树,并准备运行 Kit CLI 的独立 Node。`download_primary_runtime()` 独立下载 CPython、锁定的 Python 库、Node、pnpm 和 Office skills。两者返回绝对资源目录,并复用完整缓存。现有本地 `installPrimaryRuntime()` 操作仍只复制已有载荷,不执行下载。
|
||||
`download_office()` 下载锁定版本的 npm Kit、目标引擎及完整依赖树,并准备运行 Kit CLI 的独立 Node。`download_primary_runtime()` 独立下载 CPython、锁定的 Python 库、Node、pnpm 和 Office skills。两者返回绝对资源目录,并复用完整缓存。创作下载会在返回前校验目标平台、解释器、库目录与 skills。现有本地 `installPrimaryRuntime()` 操作仍只复制已有载荷,不执行下载。
|
||||
|
||||
`DSH_RESOURCE_CACHE` 指定绝对缓存根目录,默认为 `~/.cache/deepseek-harness/resources`。资源身份包含发布版本锁定的输入。下载会校验归档哈希和包身份、保留执行权限,并原子发布完整目录。下载失败后可重试。`DSH_RESOURCE_DOWNLOAD_TIMEOUT_MS` 限制每个归档的下载耗时,包含响应内容;默认值为 `300000`,接受 `1` 到 `2147483647` 的整数。下载遵循 Harness 代理环境策略。已安装的 wheel 文件不会被修改。
|
||||
`DSH_RESOURCE_CACHE` 指定绝对缓存根目录,默认为 `~/.cache/deepseek-harness/resources`。资源身份包含发布版本锁定的输入。下载会校验归档哈希和包身份、保留执行权限,并原子发布完整目录。下载失败后可重试。`DSH_RESOURCE_DOWNLOAD_TIMEOUT_MS` 限制每个归档的下载耗时,包含响应内容;默认值为 `300000`,接受 `1` 到 `2147483647` 的整数。下载遵循 Harness 代理环境策略。已安装的 wheel 文件不会被修改。不同发布版本使用独立缓存目录;不再使用的版本需手动删除。下载使用锁定的上游 URL,不提供 registry 或镜像覆盖。
|
||||
|
||||
后续启动时,打包 bootstrap 将已下载资源提供给 SDK profile。`DSH_PRIMARY_RUNTIME` 覆盖缓存中的创作环境;空值禁用其查询。`DSH_OFFICE_SIDECAR` 可选择包含独立 Node 的绝对 sidecar 目录。选择任一种资源即可使用 Office skills;仅有创作环境时,skills 中的 Kit CLI 被禁用。外部创作载荷保留 `primary-runtime/` 及同级 `office-skills/` 布局。SDK 原位读取资源,不复制到 `DSH_HOME`。`sdk-minimal` profile 不挂载这些提供方。
|
||||
后续启动时,打包 bootstrap 将已下载资源提供给 SDK profile。`DSH_PRIMARY_RUNTIME` 覆盖缓存中的创作环境;空值禁用其查询。`DSH_OFFICE_SIDECAR` 可选择包含独立 Node 的绝对 sidecar 目录。Office checker 随 wheel 提供;用户可自行提供 Python 环境和文档库,无需下载创作环境。选择任一种资源即可使用 Office skills;仅有创作环境时,skills 中的 Kit CLI 被禁用。外部创作载荷保留 `primary-runtime/` 及同级 `office-skills/` 布局。SDK 原位读取资源,不复制到 `DSH_HOME`。`sdk-minimal` profile 不挂载这些提供方。
|
||||
|
||||
选择 skills 与交付资源相互独立:项目、自定义目录和用户文件系统 skills 优先于同名随包 skills。SDK patch 可以仅禁用 Office 提供方,同时保留 Python 查询:
|
||||
|
||||
@@ -56,7 +56,7 @@ primary_runtime = download_primary_runtime()
|
||||
|
||||
`dsh` 在显式指定的主目录下初始化随附 profile、组合其 bundle patch,并从可执行程序的虚拟文件系统加载内置插件。运行时解析使用内存中的 generation,不创建磁盘符号链接或代理包。fallback 导入使用记录的声明包路径,包括可执行程序虚拟文件系统内的路径,因此内置配置项与外部插件 peer 共享内置的 Cordis/模块实例。原生共享库与 Windows ConPTY addon 会同其他原生 addon 一起打包;ripgrep 与 macOS PTY helper 仍是可执行伴随程序。
|
||||
|
||||
Python bootstrap 从显式下载或选择的 sidecar 解析 Office kit,让原生辅助程序与 URL Worker 使用真实文件系统路径。仓库构建与显式 Python 下载共用资源准备实现。kit 负责文档操作;Office smoke 从已加载技能获取 CLI 路径,并在空 PATH 下执行 capabilities 和 DOCX 转换。
|
||||
Python 与打包的 bootstrap 共享资源布局:`<cache>/<identity>/office/`、`node/bin/node[.exe]` 与 `node_modules/@deepseek-ai/libreoffice-kit/lib/cli.js`。自动发现 Office 缓存要求 `complete` 标记匹配清单 identity;显式 `DSH_OFFICE_SIDECAR` 要求 Node 与 CLI 文件存在。Python bootstrap 从显式下载或选择的 sidecar 解析 Office kit,让原生辅助程序与 URL Worker 使用真实文件系统路径。仓库构建与显式 Python 下载共用资源准备实现。kit 负责文档操作;Office smoke 从已加载技能获取 CLI 路径,并在空 PATH 下执行 capabilities 和 DOCX 转换。
|
||||
|
||||
外部 profile 管理使用 `dsh plugin --profile <name> ...`。该命令要求 `PATH` 中存在 `pnpm`;普通 SDK/profile 运行不需要它。
|
||||
|
||||
@@ -64,6 +64,6 @@ Python bootstrap 从显式下载或选择的 sidecar 解析 Office kit,让原
|
||||
|
||||
生产部署允许工作区中不属于运行时闭包的补丁保持未使用;闭包内包的补丁仍必须成功应用。此例外仅用于部署命令,仓库安装仍拒绝未使用的补丁。
|
||||
|
||||
在仓库根目录运行 `pnpm exec tsx scripts/build-exe-for-python-sdk.ts`,会校验闭包、构建包、部署无符号链接的文件树、打包所选目标,并把可执行程序及伴随文件同步到本模块。打包时只保留目标平台的 PTY 预构建文件,并排除运行时资源目录之外的包内 README、CHANGELOG、HISTORY Markdown 文档、source map 和 TypeScript 声明。部署会关闭工作区提升,打包时还会排除检出目录的 node_modules,防止依赖扫描把开发包加入载荷。暂存步骤让使用相同 pnpm 依赖标识的嵌套消费者共享根目录的工作区包,保留服务单例,同时保留不同的对等依赖解析。`scripts/build-python-release.py` 按仓库根版本暂存发布形态的 wheel 包,并将 `deepseek-harness-sdk` 固定到完全相同的运行时版本。
|
||||
在仓库根目录运行 `pnpm exec tsx scripts/build-exe-for-python-sdk.ts`,会校验闭包、构建包、部署无符号链接的文件树、打包所选目标,并把可执行程序及伴随文件同步到本模块。打包时只保留目标平台的 PTY 预构建文件,并排除运行时资源目录之外的包内 README、CHANGELOG、HISTORY Markdown 文档、source map 和 TypeScript 声明。部署会关闭工作区提升,打包时还会排除部署目录之外的检出依赖和工作区源码树,防止依赖扫描把开发包加入载荷。暂存步骤让使用相同 pnpm 依赖标识的嵌套消费者共享根目录的工作区包,保留服务单例,同时保留不同的对等依赖解析。`scripts/build-python-release.py` 按仓库根版本暂存发布形态的 wheel 包,并将 `deepseek-harness-sdk` 固定到完全相同的运行时版本。
|
||||
|
||||
已安装 wheel 包冒烟测试会在检出目录外创建干净的虚拟环境,验证分发物与可执行程序的来源,然后覆盖默认及自定义 SDK profile、外部插件、MCP、原生工具、直接 JSON-RPC、检入快照,以及可信运行中的真实提供方。Office 场景会迁移轻量目标载荷、显式下载 sidecar,并使用所需的平台引擎转换 DOCX:使用目标已声明的原生引擎,未声明原生引擎时使用 WASM。另见 [Python 贡献者工作流](../development.zh.md) 与 [installed-wheel 测试决策](../../.agents/notes/implemented/testing/2026-08-23-installed-python-wheel-black-box-ci.zh.md)。
|
||||
@@ -80,8 +80,11 @@ const aclRunner = process.platform === 'win32'
|
||||
? fileURLToPath(import.meta.resolve('@deepseek-ai/dsh-sandbox-windows-acl/runner'))
|
||||
: undefined
|
||||
|
||||
if (process.argv[2] === '--download-office' || process.argv[2] === '--download-primary-runtime') {
|
||||
const path = await downloadResource(process.argv[2] === '--download-office' ? 'office' : 'primary')
|
||||
const resource = process.env.DSH_RUNTIME_DOWNLOAD
|
||||
if (resource !== undefined) {
|
||||
Reflect.deleteProperty(process.env, 'DSH_RUNTIME_DOWNLOAD')
|
||||
if (resource !== 'office' && resource !== 'primary') throw new Error('DSH_RUNTIME_DOWNLOAD must select office or primary')
|
||||
const path = await downloadResource(resource)
|
||||
console.log(JSON.stringify(path))
|
||||
} else if (aclRunner !== undefined && process.argv[2] === aclRunner) {
|
||||
process.argv.splice(1, 1)
|
||||
|
||||
@@ -28,7 +28,7 @@ import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from ._resources import validate_downloads, office_launch_args
|
||||
from ._resources import validate_downloads, validate_resources, office_launch_args
|
||||
|
||||
PACKAGE_METADATA_FILENAME = "deepseek-harness-runtime.json"
|
||||
|
||||
@@ -153,8 +153,9 @@ def _node_launch_args() -> tuple[str, str]:
|
||||
return (node, str(bin_js))
|
||||
|
||||
|
||||
def _download(flag: str) -> Path:
|
||||
result = subprocess.run((*resolve_bundled_launch_args(), flag), capture_output=True, text=True)
|
||||
def _download(resource: str) -> Path:
|
||||
result = subprocess.run(resolve_bundled_launch_args(), capture_output=True, encoding="utf-8",
|
||||
env={**os.environ, "DSH_RUNTIME_DOWNLOAD": resource})
|
||||
if result.returncode:
|
||||
raise RuntimeError(result.stderr.strip() or f"resource download failed with exit code {result.returncode}")
|
||||
path = json.loads(result.stdout)
|
||||
@@ -171,7 +172,7 @@ def download_office() -> Path:
|
||||
This explicit operation requires network access on a cache miss, not a
|
||||
Harness home, agent, or model. Other SDK operations never trigger downloads.
|
||||
"""
|
||||
return _download("--download-office")
|
||||
return _download("office")
|
||||
|
||||
|
||||
def download_primary_runtime() -> Path:
|
||||
@@ -181,7 +182,9 @@ def download_primary_runtime() -> Path:
|
||||
Office engine download and does not copy resources into a Harness home.
|
||||
Completed downloads are reused; DSH_RESOURCE_CACHE selects the cache root.
|
||||
"""
|
||||
return _download("--download-primary-runtime")
|
||||
path = _download("primary")
|
||||
validate_resources(path.parent, _current_platform_tag())
|
||||
return path
|
||||
|
||||
|
||||
def resolve_office_launch_args() -> tuple[str, str]:
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Validate the authoring resources in staged, installed, or archived runtime wheels."""
|
||||
"""Validate wheel download manifests, downloaded resources, and Office process launches."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -72,6 +72,12 @@ def office_launch_args(root: Path, target: str) -> tuple[str, str]:
|
||||
office = Path(os.environ.get("DSH_OFFICE_SIDECAR", str(cache / metadata["identity"] / "office")))
|
||||
if not office.is_absolute():
|
||||
raise ValueError("DSH_OFFICE_SIDECAR must be an absolute directory")
|
||||
if "DSH_OFFICE_SIDECAR" not in os.environ:
|
||||
complete = office / "complete"
|
||||
if not complete.is_file():
|
||||
raise FileNotFoundError(f"Office download is incomplete: {office}. Call deepseek_harness_runtime.download_office() first.")
|
||||
if complete.read_text(encoding="utf-8") != metadata["identity"]:
|
||||
raise ValueError(f"Office cache identity mismatch: {office}")
|
||||
node = office / "node/bin" / ("node.exe" if target == "win-x64" else "node")
|
||||
cli = office / "node_modules/@deepseek-ai/libreoffice-kit/lib/cli.js"
|
||||
for path in (node, cli):
|
||||
@@ -80,16 +86,16 @@ def office_launch_args(root: Path, target: str) -> tuple[str, str]:
|
||||
return str(node), str(cli)
|
||||
|
||||
|
||||
def run_office_process(arguments: Sequence[str], *, capture_output: bool) -> subprocess.CompletedProcess[bytes]:
|
||||
"""Run Office with captured or inherited streams; a second interrupt kills its process tree."""
|
||||
def run_office_process(arguments: Sequence[str]) -> subprocess.CompletedProcess[bytes]:
|
||||
"""Run Office with captured streams; a second interrupt kills its process tree."""
|
||||
taskkill = os.path.join(os.environ.get("SystemRoot", ""), "System32", "taskkill.exe")
|
||||
if os.name == "nt" and not os.path.isabs(taskkill):
|
||||
raise ValueError("Office operations on Windows require an absolute SystemRoot environment value.")
|
||||
with subprocess.Popen(
|
||||
arguments,
|
||||
stdin=subprocess.DEVNULL if capture_output else None,
|
||||
stdout=subprocess.PIPE if capture_output else None,
|
||||
stderr=subprocess.PIPE if capture_output else None,
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
start_new_session=os.name != "nt",
|
||||
) as process:
|
||||
try:
|
||||
|
||||
Reference in new issue
Block a user