ci: retain shared persistent npm cache
This commit is contained in:
1 parent
4ee338532a
commit
153d54a2f2
8 files changed
+8
-45
No files matched your search
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.md
|
||||
2026-09-06-pr-ci-runner-temporary-storage.md: bab7f29604ec66a758564521e5022a577d661b93
|
||||
2026-09-06-pr-ci-runner-temporary-storage.zh.md: 5a391e2a5e9760df28007aa73941c5db84f84007
|
||||
2026-09-06-pr-ci-runner-temporary-storage.md: 0e6acd23ebf449b6b768d0c5f96ea4ba90dac2f0
|
||||
2026-09-06-pr-ci-runner-temporary-storage.zh.md: 4b1b5a76d94fdd907c5e194a2353ed5829d429ab
|
||||
@@ -12,7 +12,7 @@ The Linux failover pool runs multiple runner instances on one VM. PR coverage an
|
||||
|
||||
The static, coverage, and consumer jobs in [PR CI](../../../../.github/workflows/ci.yml) export `TMPDIR=runner.temp` through `GITHUB_ENV` in their first step before any setup or test process starts. Node, Vite, tsx, and temporary test consumers inherit the runner-owned location. Each runner owns its directory and GitHub Actions clears its removable contents at job start and completion; fixtures still allocate unique children and retain their own cleanup.
|
||||
|
||||
The three workers also set `npm_config_cache` to `runner.temp/npm-cache`. The [release workflows](../../../../.github/workflows/release.yml) apply the same cache location in their existing temporary-storage setup, including [vendor rehearsals](../../../../.github/workflows/release-vendor.yml). npm otherwise caches registry responses under the shared home directory regardless of `TMPDIR`; a temporary consumer alone does not isolate those writes. The pnpm store remains shared at `$HOME/.local/share/pnpm/store`, retaining cross-runner reuse under pnpm’s concurrent-access support. This change does not isolate its SQLite index or address capacity failures in that shared store. The consumer job also places Playwright browser downloads and installation locks beside `RUNNER_TEMP`; hosted cache restore uses that same location. Operators retain responsibility for persistent-cache capacity.
|
||||
npm keeps its configured persistent cache, normally `$HOME/.npm` on POSIX, without a per-job override in the main CI or release workflows. The pnpm store remains shared at `$HOME/.local/share/pnpm/store`. Both retain cross-runner reuse under the package managers’ concurrent-access support; shared-cache capacity and filesystem failures remain operational responsibilities. The consumer job places Playwright browser downloads and installation locks beside `RUNNER_TEMP`; hosted cache restore uses that same location.
|
||||
|
||||
The [release rehearsal decision](../process/2026-09-06-release-rehearsal-selfhosted.md) applies the same lifetime rule to release consumers. The [failover runbook](../process/2026-07-26-ci-failover-runbook.md) continues to own runner selection and shared-host capacity. This change does not retarget jobs, reduce concurrency, retry tests, weaken assertions, or modify master-only CI.
|
||||
|
||||
@@ -42,7 +42,7 @@ The advanced Python snapshot pauses only its matching workflow child’s first p
|
||||
|
||||
## Consequences
|
||||
|
||||
Output honoring these temporary-directory and cache settings follows the job lifetime instead of accumulating in unmanaged host storage. This does not reclaim existing shared temporary files, guarantee filesystem capacity, or clean files the runner account cannot remove. Operators still own historical residue, disk provisioning, and jobs outside this PR workflow.
|
||||
Output honoring `TMPDIR` follows the job lifetime instead of accumulating in unmanaged host storage. Package-manager and browser caches remain persistent. This does not reclaim existing shared temporary files, guarantee filesystem capacity, or clean files the runner account cannot remove. Operators still own historical residue, disk provisioning, and jobs outside this PR workflow.
|
||||
|
||||
Linux bwrap and Landlock workspace-write profiles grant literal `/tmp` and the workspace, not an inherited `TMPDIR` outside it; confined fixtures must place temporary writes in those granted paths. The [snapshot spill helper](../../../../packages/test-support/session-snapshot/src/harness.ts) separates fixed-length logical locators from atomically allocated live storage. A fixture-only adapter delegates saves to the real local spill provider and resolves only locators saved by that run to their live files. Recorded preview lengths, omission counts, and retrieval assertions remain unchanged; no files are allocated at the logical `/tmp/dsh-acp-snap-*` prefix. This change does not widen product sandbox grants.
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ Linux 故障切换池在同一台虚拟机上运行多个 runner 实例。PR 覆
|
||||
|
||||
[PR CI](../../../../.github/workflows/ci.yml) 的静态检查、覆盖率和消费者作业在任何准备或测试进程启动前,在首个步骤通过 `GITHUB_ENV` 导出 `TMPDIR=runner.temp`。Node、Vite、tsx 和临时测试消费者继承 runner 管理的位置。每个 runner 管理自己的目录,GitHub Actions 在作业开始和完成时清除其中可删除的内容;测试夹具仍分配唯一子目录,并保留自身清理逻辑。
|
||||
|
||||
这三个 worker 还将 `npm_config_cache` 设为 `runner.temp/npm-cache`。[发布工作流](../../../../.github/workflows/release.yml) 在既有临时存储准备步骤中采用相同缓存位置,[vendor 演练](../../../../.github/workflows/release-vendor.yml) 也如此。否则,无论 `TMPDIR` 如何设置,npm 都会在共享 home 目录中缓存注册表响应;仅使用临时消费者目录不能隔离这些写入。pnpm store 保持共享于 `$HOME/.local/share/pnpm/store`,依靠 pnpm 的并发访问支持保留跨 runner 复用。本变更不隔离其 SQLite 索引,也不解决该共享 store 的容量故障。消费者作业还将 Playwright 浏览器下载和安装锁放在 `RUNNER_TEMP` 旁;托管缓存恢复使用同一位置。持久化缓存的容量仍由运维人员负责。
|
||||
npm 保留配置的持久化缓存,在 POSIX 上通常为 `$HOME/.npm`;主 CI 和发布工作流不设置每作业覆盖。pnpm store 保持共享于 `$HOME/.local/share/pnpm/store`。两者依靠包管理器的并发访问支持保留跨 runner 复用;共享缓存容量及文件系统故障仍由运维负责。消费者作业将 Playwright 浏览器下载和安装锁放在 `RUNNER_TEMP` 旁;托管缓存恢复使用同一位置。
|
||||
|
||||
[发布演练决策](../process/2026-09-06-release-rehearsal-selfhosted.zh.md) 对发布消费者采用相同的生命周期规则。[故障切换运行手册](../process/2026-07-26-ci-failover-runbook.zh.md) 继续负责 runner 选择和共享主机容量。本变更不调整作业目标、不降低并发、不重试测试、不削弱断言,也不修改仅在 master 上执行的 CI。
|
||||
|
||||
@@ -42,7 +42,7 @@ Reference-composer 夹具将已知的 home 缩写 workspace 显示映射到既
|
||||
|
||||
## 影响
|
||||
|
||||
遵循临时目录和缓存配置的输出随作业生命周期清理,而不累积于无人管理的主机存储。本方案不回收既有共享临时文件、不保证文件系统容量,也不清理 runner 账号无权删除的文件。历史残留、磁盘配置以及本 PR 工作流以外的作业仍由运维人员负责。
|
||||
遵循 `TMPDIR` 的输出随作业生命周期清理,而不累积于无人管理的主机存储。包管理器和浏览器缓存保持持久化。本方案不回收既有共享临时文件、不保证文件系统容量,也不清理 runner 账号无权删除的文件。历史残留、磁盘配置以及本 PR 工作流以外的作业仍由运维人员负责。
|
||||
|
||||
Linux bwrap 和 Landlock 的 workspace-write profile 允许写入字面路径 `/tmp` 和 workspace,而不允许写入其外部继承的 `TMPDIR`;受限测试夹具必须将临时写入放在这些已授权路径中。[快照 spill helper](../../../../packages/test-support/session-snapshot/src/harness.ts) 将固定长度的逻辑定位符与原子分配的实际存储分开。仅用于夹具的适配器将保存操作委托给真实的本地 spill provider,并仅将本次运行已保存的定位符解析到实际文件。录制的预览长度、省略计数及检索断言保持不变;逻辑 `/tmp/dsh-acp-snap-*` 前缀下不分配文件。本变更不扩大产品沙箱授权。
|
||||
|
||||
|
||||
@@ -54,7 +54,6 @@ jobs:
|
||||
- name: Use runner-owned temporary storage
|
||||
run: |
|
||||
echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
|
||||
echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
|
||||
|
||||
# Fetch complete history so the archive gate can read the trusted PR base from a reused shallow checkout.
|
||||
- uses: actions/checkout@v6
|
||||
@@ -125,7 +124,6 @@ jobs:
|
||||
- name: Use runner-owned temporary storage
|
||||
run: |
|
||||
echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
|
||||
echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
@@ -250,7 +248,6 @@ jobs:
|
||||
- name: Use runner-owned temporary storage
|
||||
run: |
|
||||
echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
|
||||
echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
|
||||
echo "PLAYWRIGHT_BROWSERS_PATH=${RUNNER_TEMP%/*}/ms-playwright" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
@@ -53,7 +53,6 @@ jobs:
|
||||
run: |
|
||||
echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV"
|
||||
echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV"
|
||||
echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
|
||||
echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: pnpm/action-setup@v4
|
||||
|
||||
@@ -50,7 +50,6 @@ jobs:
|
||||
run: |
|
||||
echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV"
|
||||
echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV"
|
||||
echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
|
||||
echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: pnpm/action-setup@v4
|
||||
@@ -113,7 +112,6 @@ jobs:
|
||||
run: |
|
||||
echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV"
|
||||
echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV"
|
||||
echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
|
||||
echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: pnpm/action-setup@v4
|
||||
|
||||
@@ -45,7 +45,6 @@ describe('CI workflow', () => {
|
||||
name: 'Use runner-owned temporary storage',
|
||||
run: [
|
||||
'echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"',
|
||||
'echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"',
|
||||
...(jobName === 'node-24-consumers'
|
||||
? ['echo "PLAYWRIGHT_BROWSERS_PATH=${RUNNER_TEMP%/*}/ms-playwright" >> "$GITHUB_ENV"']
|
||||
: []),
|
||||
|
||||
@@ -9,7 +9,6 @@ const root = resolve(import.meta.dirname, '../..')
|
||||
const repository = 'deepseek-harness/deepseek-harness'
|
||||
const selfhosted = ['self-hosted', 'linux', 'x64', 'vm-backup']
|
||||
const hosted = 'ubuntu-24.04'
|
||||
const npmCacheExport = 'echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"'
|
||||
|
||||
interface Step {
|
||||
name?: string
|
||||
@@ -39,14 +38,6 @@ function evaluate(expression: string, context: Record<string, string | boolean>)
|
||||
return runInNewContext(source, { fromJSON: JSON.parse }, { timeout: 1000 }) as unknown
|
||||
}
|
||||
|
||||
function assertEarlyNpmCacheExport(steps: Step[]): void {
|
||||
const cacheIndex = steps.findIndex(step => step.run?.split('\n').includes(npmCacheExport))
|
||||
const pnpmIndex = steps.findIndex(step => step.uses?.startsWith('pnpm/') || /\bpnpm\b/.test(step.run ?? ''))
|
||||
expect(cacheIndex).toBeGreaterThanOrEqual(0)
|
||||
expect(cacheIndex).toBeLessThan(pnpmIndex)
|
||||
expect(steps[cacheIndex]?.if).toBeUndefined()
|
||||
}
|
||||
|
||||
function assertSharedPersistentStore(run: string | undefined): void {
|
||||
expect(run).toContain('store_root="$HOME/.local/share/pnpm/store"')
|
||||
expect(run).toContain('echo "PNPM_CONFIG_STORE_DIR=$store_root" >> "$GITHUB_ENV"')
|
||||
@@ -123,29 +114,8 @@ for (const [file, jobIds] of [['release.yml', ['dependencies', 'pack']], ['relea
|
||||
.toBe('${{ runner.temp }}/setup-pnpm-${{ github.run_id }}-${{ github.run_attempt }}-${{ github.job }}')
|
||||
expect(job.steps.find(step => step.name === 'Install (immutable)')?.run).toBe('pnpm install --frozen-lockfile')
|
||||
})
|
||||
it('exports a runner-private npm cache before package-manager setup', () => {
|
||||
assertEarlyNpmCacheExport(job.steps)
|
||||
})
|
||||
it.each([
|
||||
['missing', ''],
|
||||
['shared home', 'echo "npm_config_cache=$HOME/.npm" >> "$GITHUB_ENV"'],
|
||||
['step-local', 'export npm_config_cache="${{ runner.temp }}/npm-cache"'],
|
||||
])('rejects a %s npm cache export', (_name, replacement) => {
|
||||
const steps = job.steps.map(step => step.run === undefined
|
||||
? step
|
||||
: { ...step, run: step.run.replace(npmCacheExport, replacement) })
|
||||
expect(() => { assertEarlyNpmCacheExport(steps) }).toThrow()
|
||||
})
|
||||
it('rejects a conditional npm cache export', () => {
|
||||
const steps = job.steps.map(step => step.run?.includes(npmCacheExport) ? { ...step, if: 'false' } : step)
|
||||
expect(() => { assertEarlyNpmCacheExport(steps) }).toThrow()
|
||||
})
|
||||
it('rejects an npm cache export after package-manager setup', () => {
|
||||
const steps = job.steps.map(step => step.run === undefined
|
||||
? step
|
||||
: { ...step, run: step.run.replace(npmCacheExport, '') })
|
||||
steps.push({ run: npmCacheExport })
|
||||
expect(() => { assertEarlyNpmCacheExport(steps) }).toThrow()
|
||||
it('retains the configured shared npm cache', () => {
|
||||
expect(JSON.stringify(release)).not.toMatch(/npm_config_cache/i)
|
||||
})
|
||||
it.each(['', 'store_root="${RUNNER_TEMP%/*}/pnpm-store"', 'store_root="$RUNNER_TEMP/pnpm-store"'])(
|
||||
'rejects missing, runner-private, or job-temporary store placement: %s', (replacement) => {
|
||||
|
||||
Reference in new issue
Block a user