diff --git a/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.i18n.yaml index 5be7e083a3..91613aea4a 100644 --- a/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.i18n.yaml @@ -33,8 +33,8 @@ en: a244f6a88fdabf3d zh: e4feedb7297f4865 /agent-note-typert-gateway-targeted-method-calls/three-artifact-kinds-and-two-typescript-programs: - en: 6ea4bddcf35a834d - zh: 4af474c6818b1cf0 + en: a5134745815f762f + zh: 99b6a905abd58e6a /agent-note-typert-gateway-targeted-method-calls/the-remote-package-entry: en: 75e977fe0fb71d59 zh: 00760c79da173cb7 diff --git a/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.md b/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.md index 58135ff56e..360e3affe9 100644 --- a/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.md +++ b/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.md @@ -215,7 +215,7 @@ Host lib build → Vite 构建 Web ``` -The existing top-level `build` still runs `build:lib` before `build:web`, but `build:lib` must complete the Host and Remote artifacts before starting Client TypeScript compilation. A clean build must not depend on stale `.d.ts` files from an earlier build. +The top-level `build` runs the step graph in `scripts/build.ts`, which completes the Host bundle and the Remote artifacts before starting Client TypeScript compilation. `build:lib` remains the complete serial lib build for `build:bench`, and its Host half `build:lib:host` serves `typecheck` and the vendor release workflows. A clean build must not depend on stale `.d.ts` files from an earlier build. Compiler-backed repository gates that resolve the consumer surface have the same prerequisite even when their primary inputs are source files. The public `typecheck`, `lint`, and `doc-typecheck` commands run the Host contract pass first. The gate scheduler may use their `*:contracts-ready` variants only after an explicit Typert-contract or complete-build dependency, so parallel lanes neither read missing declarations nor run concurrent generators against the same outputs. diff --git a/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.zh.md b/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.zh.md index 21d51800ae..51b08341c2 100644 --- a/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.zh.md @@ -215,7 +215,7 @@ Host lib build → Vite 构建 Web ``` -现有顶层 `build` 仍表现为先 `build:lib`、再 `build:web`,但 `build:lib` 内部必须先完成 Host 与 Remote artifact,再启动 Client TypeScript 编译。一次干净构建不能依赖上次残留的 `.d.ts`。 +顶层 `build` 运行 `scripts/build.ts` 中的步骤图,先完成 Host bundle 与 Remote artifact,再启动 Client TypeScript 编译。`build:lib` 仍是 `build:bench` 使用的完整串行 lib 构建,其 Host 部分 `build:lib:host` 供 `typecheck` 与 vendor 发布 workflow 使用。一次干净构建不能依赖上次残留的 `.d.ts`。 即使主要输入是源文件,需要通过编译器解析消费方 surface 的仓库门禁也有相同的前置条件。公共 `typecheck`、`lint` 和 `doc-typecheck` 命令会先执行 Host 约定 pass。门禁调度器仅可在显式的 Typert 约定依赖或完整构建依赖完成后使用对应的 `*:contracts-ready` 变体,使并行 lane 既不会读取缺失的声明,也不会针对同一输出并发运行多个生成器。 diff --git a/.agents/notes/implemented/process/2026-10-11-build-src-test-split.i18n.yaml b/.agents/notes/implemented/process/2026-10-11-build-src-test-split.i18n.yaml index 2e80be158f..8a1cab5a29 100644 --- a/.agents/notes/implemented/process/2026-10-11-build-src-test-split.i18n.yaml +++ b/.agents/notes/implemented/process/2026-10-11-build-src-test-split.i18n.yaml @@ -6,8 +6,8 @@ en: db08cc8c6fbc40cc zh: 740e25dede205595 /agent-note-split-build-src-and-build-test-with-build-running-both-concurrently/problem: - en: 3c0f9561851590cd - zh: 01510bc8e2dd4a44 + en: bc382ad1f1e71352 + zh: 44189bf0f69a0bf7 /agent-note-split-build-src-and-build-test-with-build-running-both-concurrently/decision: en: a15c7c13f167e5ac zh: 63bcc2ef60c2f344 @@ -21,20 +21,20 @@ en: 5872f92bf06cb776 zh: 274fcfbd971d5c5b /agent-note-split-build-src-and-build-test-with-build-running-both-concurrently/decision/callers: - en: 79559fef50ac3f1f - zh: bf327d9b03d4bcf4 + en: 932a69653dff2a17 + zh: 7ceac59cb9abc6f9 /agent-note-split-build-src-and-build-test-with-build-running-both-concurrently/decision/measurements: - en: 5481a604777b29d7 - zh: 63e330b736f6c7a3 + en: 53f9a558836c917c + zh: b372b301e5a9f23b /agent-note-split-build-src-and-build-test-with-build-running-both-concurrently/decision/completeness-and-lossless-evidence: - en: a65438822fc683b2 - zh: 34d5595110955b8e + en: 2d03c309e077c0de + zh: a8ce74fbc67fff6c /agent-note-split-build-src-and-build-test-with-build-running-both-concurrently/decision/intentional-trade-offs-not-lossless: - en: 7e572606c676e46f - zh: f75615dd4a481391 + en: 6a6229e5c7fbda91 + zh: e38c0596e0fe184c /agent-note-split-build-src-and-build-test-with-build-running-both-concurrently/alternatives-considered: en: d9492c56ba2a210a zh: feafcad4f632a057 /agent-note-split-build-src-and-build-test-with-build-running-both-concurrently/consequences: - en: 8c4e6eec5bb44b5a - zh: 3f86eb0e620b62ff + en: 4b16a5b02479f4ff + zh: d75672318343a743 diff --git a/.agents/notes/implemented/process/2026-10-11-build-src-test-split.md b/.agents/notes/implemented/process/2026-10-11-build-src-test-split.md index ae9e93bd0c..c4aede7339 100644 --- a/.agents/notes/implemented/process/2026-10-11-build-src-test-split.md +++ b/.agents/notes/implemented/process/2026-10-11-build-src-test-split.md @@ -6,7 +6,7 @@ English | [中文](2026-10-11-build-src-test-split.zh.md) ## Problem -`pnpm run build` runs the native addon, the package-library build (Host `tsc -b tsconfig.host.json` → Host tsdown → Desktop bundle → Client `tsc -b tsconfig.client.json` → Client tsdown), and the Web build serially. Besides referencing package projects, the Host and Client aggregates each carry a `noEmit` root program covering tests, scripts, benchmarks, and website. These two root programs emit no files and feed no later artifact step, yet sit on the serial critical path: timed alone, the Host root program takes 68.5 s and the Client root program 36.0 s, while the serial package-library build takes 247.3–251.4 s. +`pnpm run build` runs the native addon, the package-library build (Host `tsc -b tsconfig.host.json` → Host tsdown → Desktop bundle → Client `tsc -b tsconfig.client.json` → Client tsdown), and the Web build serially. Besides referencing package projects, the Host and Client aggregates each carry a `noEmit` root program covering tests, scripts, benchmarks, and website. These two root programs emit no files and feed no later artifact step, yet sit on the serial critical path: timed alone, the Host root program takes 68.5 s and the Client root program 36.0 s, while the serial package-library build takes about 249 s (median of 2). Development iterations need only artifacts; test and script type checks are needed before work is declared complete. The only way to skip the two root programs is the CI-oriented `pnpm run build --artifacts-only`, which has no local development command name and no counterpart that runs only those two checks. Meanwhile, `pnpm run build` must stay complete and detect every type error. @@ -75,7 +75,7 @@ native → host-src ─┬─ host-test ───────────── | PR rehearsal in `release-vendor.yml` | `compile-referenced-projects.ts host-libraries` | None | | Non-PR branch of `release-vendor.yml`, `release-vendor-publish.yml` | `build:lib:host` | Host aggregate root program only | -`lint`, `lint:fix`, `doc-typecheck`, and the Typert contracts gate read only Typert outputs and package declarations: both aggregates are `noEmit`, oxlint builds its own programs, `doc-typecheck` references only package projects, and no consumer of `host-libraries` reads `apps/desktop/lib`. Their prerequisite step therefore saves about 68 s of Host aggregate checking and about 4.4 s of Desktop bundling on every run. +`lint`, `lint:fix`, `doc-typecheck`, and the Typert contracts gate read only Typert outputs and package declarations: both aggregates are `noEmit`, oxlint builds its own programs, `doc-typecheck` references only package projects, and no consumer of `host-libraries` reads `apps/desktop/lib`. Their prerequisite step therefore skips the Host aggregate check and the Desktop bundle and falls from 159.8 s to 90.9 s (see the measurements below). The consumers lane's HMR Web test starts `dev:web` and therefore uses `build:src` inside that lane; the same lane's `build` gate still runs both aggregate checks. CI does not call `build:test` separately: the two required lanes already get the same checks through `build`. The [PR artifact build decision](2026-10-05-pr-artifact-typechecks.md) keeps its required complete builds, benchmark `--noCheck`, job and worker ceilings, serial node-compat Vitest, coverage weights, and pnpm packing checks; its PR artifact build is `build:src`. @@ -83,27 +83,27 @@ The consumers lane's HMR Web test starts `dev:web` and therefore uses `build:src Prototype conditions: a shared 256 vCPU Linux host (load average 23–70), Node v24.18.0, TypeScript 6.0.3, React type packages hoisted to the root, without the Typert changes of [the lossless speed-ups note](2026-10-11-lossless-build-speedups.md); scope is the package-library build (excluding native and web). Every round starts from `pnpm run clean`, running once each in the order serial, concurrent, `build:lib`, `build:lib`, concurrent, serial; `/usr/bin/time -v` times each step, and summed process-tree RSS is sampled every 250 ms. The prototype runs the aggregate checks as `tsc -b tsconfig.{host,client}.json` once references are up to date, when build mode builds only the aggregate root program. -| Mode | Wall (s) | CPU user+sys (s) | Process-tree RSS peak (GiB) | +| Mode (medians of 2) | Wall (s) | CPU user+sys (s) | Process-tree RSS peak (GiB) | |---|---|---|---| -| Serial `pnpm run build:lib` | 247.3 / 251.4 | 370.8 / 385.1 | 5.90 / 5.99 | -| Each step run separately in sequence | 269.1 / 241.1 | 377.1 / 370.4 | 5.93 / 5.70 | -| Concurrent by dependency graph | 178.0 / 166.2 | 397.4 / 386.8 | 7.99 / 8.01 | +| Serial `pnpm run build:lib` | 249.4 | 378.0 | 5.95 | +| Each step run separately in sequence | 255.1 | 373.8 | 5.82 | +| Concurrent by dependency graph | 172.1 | 392.1 | 8.00 | -Median wall falls from 249.4 s to 172.1 s (−77.3 s, −31%); CPU grows by about 14 s (+3.7%) because the Host aggregate check's user time rises from 98–100 s to 111–124 s under contention; the RSS peak grows by 2.1 GiB where the Host aggregate check (3.95 GiB) overlaps Host tsdown (5.65 GiB). The concurrent critical path is host-src → host-bundle → desktop-bundle → client-src → client-test; host-test finishes 32.5–33.1 s before the end and client-bundle about 21 s before. Per-step wall (two serial / two concurrent): host-src 73.8, 61.3 / 70.3, 62.6; host-test 68.5, 64.0 / 75.1, 70.5; host-bundle 23.1, 20.5 / 22.1, 21.0; desktop-bundle 4.5, 4.0 / 4.4, 4.4; client-src 50.4, 46.3 / 46.3, 44.1; client-test 36.0, 33.5 / 34.9, 34.1; client-bundle 12.9, 11.5 / 13.6, 12.6. +Median wall falls from 249.4 s to 172.1 s (−77.3 s, −31%). The two runs of each step separately differ by 28 s, so the overhead of running the steps as separate processes is not resolvable. CPU grows by about 14 s (+3.7%), mostly Host aggregate check user time under contention; the RSS peak grows by about 2.1 GiB where the Host aggregate check (about 4 GiB) overlaps Host tsdown (about 5.7 GiB). The concurrent critical path is host-src → host-bundle → desktop-bundle → client-src → client-test; host-test finishes about 33 s before the end and client-bundle about 21 s before. -Round 1 final measurements: one sample per scenario, each from clean, in a 64-core cgroup (load average 29–87); the baseline is master. +Final measurements: master (the merge base) and this branch run interleaved, each from clean, in a 64-core cgroup; values are medians of 2 samples unless marked n=1. | Scenario | master | This branch | CPU user+sys (master → branch) | Process-tree RSS peak (master → branch) | |---|---|---|---|---| -| Complete build `pnpm run build` | 318.2 s | 170.1 s (−46.5%) | 425.5 → 373.4 s | 6.13 → 7.55 GiB | -| Artifact build: master `build --artifacts-only`, branch `build:src` | 178.9 s | 152.5 s (−14.8%) | Not measured | Not measured | -| Development iteration: master `build`, branch `build:src` | 318.2 s | 152.5 s (−52%) | Not measured | Not measured | -| `pnpm run build:test` (after `build:src`) | — | 67.9 s | Not measured | Not measured | -| `pnpm run build:test` immediate rerun | — | 19.4 s | Not measured | Not measured | -| `DSH_GATE_CONCURRENCY=1 pnpm run build` | — | Round 2 pending | Round 2 pending | Round 2 pending | -| Prerequisite step of `lint` and the others: master `build:lib:host`, branch `host-libraries` | Round 2 pending | Round 2 pending | Round 2 pending | Round 2 pending | +| Complete build `pnpm run build` | 317.1 s | 169.1 s (−46.7%) | 423.1 → 374.4 s (−11.5%) | 6.11 → 7.53 GiB (+1.42) | +| Serial branch build `DSH_GATE_CONCURRENCY=1 pnpm run build` (n=1) | — | 266.7 s | 389.5 s | 6.27 GiB | +| Artifact build: master `build --artifacts-only`, branch `build:src` | 185.8 s | 153.6 s (−17.3%) | 256.2 → 231.2 s | 6.58 → 6.18 GiB | +| Development iteration: master `build`, branch `build:src` | 317.1 s | 153.6 s (−51.6%) | 423.1 → 231.2 s (−45.4%) | 6.11 → 6.18 GiB | +| `pnpm run build:test` after `build:src` | — | 67.7 s | 144.6 s | 5.33 GiB | +| `pnpm run build:test` no-op rerun | — | 19.4 s | 33.1 s | Not measured | +| Prerequisite of `lint` and `doc-typecheck`: master `build:lib:host`, branch `host-libraries` | 159.8 s | 90.9 s (−43.2%) | 234.7 → 136.0 s | Not measured | -Step walls of this branch's `build`: native 1.7, host-src 66.5, host-bundle 11.4, desktop-bundle 3.4, client-src 45.6, client-bundle 9.0, web 9.4, host-test 65.3, client-test 34.8 s. From these values the critical path is native → host-src → host-bundle → desktop-bundle → client-src → client-test, 163.4 s in total; host-test ends at 133.5 s and is off the critical path. master's serial segments: native 2.5, Host package compilation 66.7, Host aggregate check 66.1, Host tsdown 22.5, Desktop bundle 3.3, Client package compilation 55.6, Client aggregate check 68.2, Client tsdown 13.9, web 9.7 s, 308.5 s in total. The difference also includes the changes of [the lossless speed-ups note](2026-10-11-lossless-build-speedups.md): the shorter Host tsdown, Client package compilation, Client tsdown, and Client aggregate check come from that note, while moving the Host aggregate check off the critical path comes from this note's concurrent scheduling. Within `build:test`'s 67.9 s, the Host and Client aggregate checks run concurrently, taking 65.2 s and 33.9 s. +The serial branch sample overlapped other work on the host (742 s of foreign CPU), so its wall likely overstates the serial cost. In the serial build segments (master → branch, one sample each), Host tsdown falls from 22.5 to 11.7 s, Client package compilation from 55.6 to 48.7 s, the Client aggregate check from 68.2 to 35.9 s, and Client tsdown from 13.9 to 10.5 s; the segments sum to 308.5 s and 254.9 s. Those reductions come from [the lossless speed-ups note](2026-10-11-lossless-build-speedups.md); the further reduction from 254.9 s of serial segments to the 169.1 s concurrent build comes from this note's scheduling, which takes the Host aggregate check (about 66 s) off the critical path. In the concurrent build, the critical path is native → host-src → host-bundle → desktop-bundle → client-src → client-test, and host-test ends about 30 s before the last step (one earlier sample). `build:test` runs the Host and Client aggregate checks concurrently, so its wall is the Host check's (about 65 s). ### Completeness and lossless evidence @@ -119,14 +119,16 @@ Step walls of this branch's `build`: native 1.7, host-src 66.5, host-bundle 11.4 Negative controls: after a type error (TS2322) is added to `scripts/pnpm-invocation.spec.ts`, which only the Host aggregate root program contains, `build:src` exits 0; `build:test` exits 1 with the Host aggregate check failing and the Client aggregate check passing; `build` exits 1 with 8 of 9 steps passing and 1 failing, and writes no client build record. In a scratch fixture whose referenced project's tsconfig has an invalid `target`, a JSON syntax error, or an unknown compiler option, `tsc -p` exits 0, while `tsc -b` reports TS6046, TS5092 with TS1012, and TS5023 respectively and exits 1; when the referenced project lacks `composite`, both report TS6306. +CI evidence: on PR #6005 both required lanes that run the concurrent `pnpm run build` passed, `node 24 / snapshots and artifacts` (`check:ci:consumers` on `blacksmith-16vcpu-ubuntu-2404`) and `windows node 24 / build & observational` (`check:ci:windows-blocking` on `blacksmith-16vcpu-windows-2025`); the GitHub-hosted 4 vCPU benchmarks lane does not run `build`. The inner concurrency follows `DSH_GATE_CONCURRENCY`: master serial lanes set 1, the consumers lane sets 10 under self-hosted failover, and an unset value selects the available CPU count, capped at the 9 steps. + ### Intentional trade-offs (not lossless) - **Some commands skip the aggregate checks:** `build:src`, the launch builds of `dev:web` and `dev:desktop`, and the prerequisite steps of `lint`, `lint:fix`, `doc-typecheck`, and the Typert contracts gate do not type-check the two aggregate root programs, and these prerequisite steps do not produce `apps/desktop/lib`. `build`, `build:test`, `typecheck` (pre-push hook), and the two required CI lanes own completeness; `dev:web` and `dev:desktop` do not refuse to launch because of test type errors. - **`build:test` does not check freshness:** When sources change after `build:src`, `build:test` checks against stale declarations, including Typert Remote declarations; rerun `build:src` first. The complete `build` is unaffected. -- **Concurrent `build` raises the memory peak:** This branch's complete `build` peaks at 7.55 GiB of process-tree RSS against 6.13 GiB for the master serial build (+1.42 GiB, one sample each); CPU meanwhile falls from 425.5 s to 373.4 s because [the lossless speed-ups note](2026-10-11-lossless-build-speedups.md) removes more work than concurrent contention adds. In the prototype (without the Typert changes), concurrent scheduling alone raises the RSS peak by about 2.1 GiB (5.9 → 8.0 GiB) and CPU by about 3.7%. `DSH_GATE_CONCURRENCY` bounds it; under memory pressure use `DSH_GATE_CONCURRENCY=1 pnpm run build`, or run `build:src` and `build:test` separately. In the windows-build lane, `build` runs concurrently with `docs:build`. +- **Concurrent `build` raises the memory peak:** This branch's complete `build` peaks at 7.53 GiB of process-tree RSS against 6.11 GiB for the master serial build (+1.42 GiB, medians of 2) and 6.27 GiB for the branch's serial build (n=1); CPU meanwhile falls from 423.1 s to 374.4 s because [the lossless speed-ups note](2026-10-11-lossless-build-speedups.md) removes more work than concurrent contention adds. In the prototype (without the Typert changes), concurrent scheduling alone raises the RSS peak by about 2.1 GiB (5.9 → 8.0 GiB) and CPU by about 3.7%. `DSH_GATE_CONCURRENCY` bounds it; under memory pressure use `DSH_GATE_CONCURRENCY=1 pnpm run build`, or run `build:src` and `build:test` separately. In the windows-build lane, `build` runs concurrently with `docs:build`. - **Passing steps' output is hidden by default:** Only failed steps print their complete output. -- **`tsc -p` does not skip an up-to-date root program:** An immediate rerun of `build:test` without changes still takes 19.4 s because both programs are parsed again; `tsc -b` skips an up-to-date root. -- **On many-core cold builds `build:src` is only 17.6 s faster than `build`:** This branch's cold `build` takes 170.1 s and `build:src` 152.5 s (one sample each), because host-test is off the critical path and client-test (34.8 s) is only about 16 s longer than client-bundle plus web (18.4 s). The rest of `build:src`'s saving is CPU and memory: in the prototype the two aggregate checks together use about 148 s of CPU, and the Host aggregate check overlapping Host tsdown raises the memory peak by about 2 GiB; on few-core machines and in incremental rebuilds this also turns into wall time. +- **`tsc -p` does not skip an up-to-date root program:** An immediate rerun of `build:test` without changes still takes 19.4 s and 33.1 s of CPU because both programs are parsed again; `tsc -b` skips an up-to-date root. +- **On many-core cold builds `build:src` is only 15.5 s faster than `build`:** This branch's cold `build` takes 169.1 s and `build:src` 153.6 s (medians of 2), because host-test is off the critical path and client-test is only about 16 s longer than client-bundle plus web. The rest of `build:src`'s saving is CPU and memory: 374.4 → 231.2 s of CPU and 7.53 → 6.18 GiB of peak RSS; on few-core machines and in incremental rebuilds this also turns into wall time. - **CI invokes public commands in the root manifest:** PR artifact builds invoke `build:src` rather than an internal helper flag; a PR that edits these root manifest scripts activates the separate Node Addon System platform matrix. ## Alternatives considered @@ -144,7 +146,7 @@ Negative controls: after a type error (TS2322) is added to `scripts/pnpm-invocat ## Consequences -Development iterations can skip the two aggregate root programs with `build:src` (52% faster cold than master's `build`) and complete the checks with `build:test` or `build` before declaring work complete. `build` stays complete: the Host aggregate check is off the critical path, the Client aggregate check runs alongside the Client bundle and web, and a cold build is 46.5% faster than master (including the lossless speed-ups note's changes; one sample each). The costs are a higher memory peak, a little extra CPU from concurrent contention, less passing output by default, and several development and prerequisite commands that do not check test types (see the trade-offs above). +Development iterations can skip the two aggregate root programs with `build:src` (51.6% faster cold than master's `build`) and complete the checks with `build:test` or `build` before declaring work complete. `build` stays complete: the Host aggregate check is off the critical path, the Client aggregate check runs alongside the Client bundle and web, and a cold build is 46.7% faster than master (including the lossless speed-ups note's changes; medians of 2). The costs are a higher memory peak, a little extra CPU from concurrent contention, less passing output by default, and several development and prerequisite commands that do not check test types (see the trade-offs above). Removing or changing `build` in the two required lanes, consumers and windows-build, requires first transferring both aggregate checks to another blocking owner. If Typert later gains its own generation step, the step table adds it and client-src depends on it; command semantics and scheduling stay the same. diff --git a/.agents/notes/implemented/process/2026-10-11-build-src-test-split.zh.md b/.agents/notes/implemented/process/2026-10-11-build-src-test-split.zh.md index fd659bbf63..ed7e1f6344 100644 --- a/.agents/notes/implemented/process/2026-10-11-build-src-test-split.zh.md +++ b/.agents/notes/implemented/process/2026-10-11-build-src-test-split.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -`pnpm run build` 串行执行 native addon、包库构建(Host `tsc -b tsconfig.host.json` → Host tsdown → Desktop bundle → Client `tsc -b tsconfig.client.json` → Client tsdown)和 Web 构建。Host 与 Client 聚合配置除了引用各包 project,还各自带有一个 `noEmit` 根程序,覆盖 tests、scripts、benchmarks 与 website。这两个根程序不产出任何文件,也不是后续产物步骤的输入,却位于串行关键路径上:单独计时时 Host 根程序为 68.5 s、Client 根程序为 36.0 s,串行包库构建为 247.3–251.4 s。 +`pnpm run build` 串行执行 native addon、包库构建(Host `tsc -b tsconfig.host.json` → Host tsdown → Desktop bundle → Client `tsc -b tsconfig.client.json` → Client tsdown)和 Web 构建。Host 与 Client 聚合配置除了引用各包 project,还各自带有一个 `noEmit` 根程序,覆盖 tests、scripts、benchmarks 与 website。这两个根程序不产出任何文件,也不是后续产物步骤的输入,却位于串行关键路径上:单独计时时 Host 根程序为 68.5 s、Client 根程序为 36.0 s,串行包库构建约为 249 s(2 个样本的中位数)。 开发迭代只需要产物,测试与脚本的类型检查在声明完成前才需要。能跳过两个根程序的只有面向 CI 的 `pnpm run build --artifacts-only`;它没有本地开发命令名,也没有只做这两项检查的对应命令。同时,`pnpm run build` 必须保持完整,能检测出所有类型错误。 @@ -75,7 +75,7 @@ native → host-src ─┬─ host-test ───────────── | `release-vendor.yml` 的 PR 演练 | `compile-referenced-projects.ts host-libraries` | 无 | | `release-vendor.yml` 的非 PR 分支、`release-vendor-publish.yml` | `build:lib:host` | 仅 Host 聚合根程序 | -`lint`、`lint:fix`、`doc-typecheck` 与 Typert contracts gate 只读取 Typert 输出与包声明:两个聚合配置都是 `noEmit`,oxlint 自行构建程序,`doc-typecheck` 只引用包 project,`host-libraries` 的消费方都不读取 `apps/desktop/lib`。它们的前置步骤因此每次省去约 68 s 的 Host 聚合检查与约 4.4 s 的 Desktop bundle。 +`lint`、`lint:fix`、`doc-typecheck` 与 Typert contracts gate 只读取 Typert 输出与包声明:两个聚合配置都是 `noEmit`,oxlint 自行构建程序,`doc-typecheck` 只引用包 project,`host-libraries` 的消费方都不读取 `apps/desktop/lib`。它们的前置步骤因此跳过 Host 聚合检查与 Desktop bundle,耗时从 159.8 s 降到 90.9 s(见下文测量)。 consumers lane 的 HMR Web 测试启动 `dev:web`,因此在该 lane 内使用 `build:src`;同一 lane 的 `build` gate 仍运行两个聚合检查。CI 不单独调用 `build:test`:两个必需 lane 已通过 `build` 获得同一组检查。[PR 产物构建决策](2026-10-05-pr-artifact-typechecks.zh.md)的必需完整构建、benchmark `--noCheck`、job 与 worker 上限、node-compat 串行 Vitest、覆盖率权重与 pnpm 打包检查仍然有效;其中的 PR 产物构建由 `build:src` 执行。 @@ -83,27 +83,27 @@ consumers lane 的 HMR Web 测试启动 `dev:web`,因此在该 lane 内使用 原型测量的条件:256 vCPU 共享 Linux 主机(load average 23–70),Node v24.18.0、TypeScript 6.0.3,React 类型包已提升到根,不含[无损提速 Note](2026-10-11-lossless-build-speedups.zh.md) 的 Typert 改动;范围是包库构建(不含 native 与 web)。每轮从 `pnpm run clean` 开始,按 串行、并发、`build:lib`、`build:lib`、并发、串行 的顺序各运行一次;每个步骤由 `/usr/bin/time -v` 计时,进程树 RSS 每 250 ms 采样求和。原型中的聚合检查以 `tsc -b tsconfig.{host,client}.json` 在 references 已是最新时运行,此时 build mode 只构建聚合根程序。 -| 方式 | wall(s) | CPU user+sys(s) | 进程树 RSS 峰值(GiB) | +| 方式(2 个样本的中位数) | wall(s) | CPU user+sys(s) | 进程树 RSS 峰值(GiB) | |---|---|---|---| -| 串行 `pnpm run build:lib` | 247.3 / 251.4 | 370.8 / 385.1 | 5.90 / 5.99 | -| 各步骤依次单独运行 | 269.1 / 241.1 | 377.1 / 370.4 | 5.93 / 5.70 | -| 按依赖图并发 | 178.0 / 166.2 | 397.4 / 386.8 | 7.99 / 8.01 | +| 串行 `pnpm run build:lib` | 249.4 | 378.0 | 5.95 | +| 各步骤依次单独运行 | 255.1 | 373.8 | 5.82 | +| 按依赖图并发 | 172.1 | 392.1 | 8.00 | -wall 中位数从 249.4 s 降到 172.1 s(−77.3 s,−31%);CPU 增加约 14 s(+3.7%),来自争用下 Host 聚合检查的 user 时间从 98–100 s 增至 111–124 s;RSS 峰值增加 2.1 GiB,出现在 Host 聚合检查(3.95 GiB)与 Host tsdown(5.65 GiB)重叠时。并发时的关键路径是 host-src → host-bundle → desktop-bundle → client-src → client-test;host-test 在结束前 32.5–33.1 s 完成,client-bundle 在结束前约 21 s 完成。各步骤 wall(串行两次 / 并发两次):host-src 73.8、61.3 / 70.3、62.6;host-test 68.5、64.0 / 75.1、70.5;host-bundle 23.1、20.5 / 22.1、21.0;desktop-bundle 4.5、4.0 / 4.4、4.4;client-src 50.4、46.3 / 46.3、44.1;client-test 36.0、33.5 / 34.9、34.1;client-bundle 12.9、11.5 / 13.6、12.6。 +wall 中位数从 249.4 s 降到 172.1 s(−77.3 s,−31%)。各步骤单独运行的两次样本相差 28 s,因此把步骤拆成独立进程的开销无法分辨。CPU 增加约 14 s(+3.7%),主要是争用下 Host 聚合检查增加的 user 时间;RSS 峰值增加约 2.1 GiB,出现在 Host 聚合检查(约 4 GiB)与 Host tsdown(约 5.7 GiB)重叠时。并发时的关键路径是 host-src → host-bundle → desktop-bundle → client-src → client-test;host-test 在结束前约 33 s 完成,client-bundle 在结束前约 21 s 完成。 -第 1 轮最终测量:每个场景一个样本,均从 clean 开始,在 64 核 cgroup 中运行(load average 29–87);基线为 master。 +最终测量:master(合并基点)与本分支交替运行,每次均从 clean 开始,在 64 核 cgroup 中运行;除标注 n=1 者外,数值均为 2 个样本的中位数。 | 场景 | master | 本分支 | CPU user+sys(master → 本分支) | 进程树 RSS 峰值(master → 本分支) | |---|---|---|---|---| -| 完整构建 `pnpm run build` | 318.2 s | 170.1 s(−46.5%) | 425.5 → 373.4 s | 6.13 → 7.55 GiB | -| 产物构建:master `build --artifacts-only`,本分支 `build:src` | 178.9 s | 152.5 s(−14.8%) | 未测量 | 未测量 | -| 开发迭代:master `build`,本分支 `build:src` | 318.2 s | 152.5 s(−52%) | 未测量 | 未测量 | -| `pnpm run build:test`(`build:src` 之后) | — | 67.9 s | 未测量 | 未测量 | -| `pnpm run build:test` 立即重跑 | — | 19.4 s | 未测量 | 未测量 | -| `DSH_GATE_CONCURRENCY=1 pnpm run build` | — | 第 2 轮待测 | 第 2 轮待测 | 第 2 轮待测 | -| `lint` 等的前置步骤:master `build:lib:host`,本分支 `host-libraries` | 第 2 轮待测 | 第 2 轮待测 | 第 2 轮待测 | 第 2 轮待测 | +| 完整构建 `pnpm run build` | 317.1 s | 169.1 s(−46.7%) | 423.1 → 374.4 s(−11.5%) | 6.11 → 7.53 GiB(+1.42) | +| 本分支串行构建 `DSH_GATE_CONCURRENCY=1 pnpm run build`(n=1) | — | 266.7 s | 389.5 s | 6.27 GiB | +| 产物构建:master `build --artifacts-only`,本分支 `build:src` | 185.8 s | 153.6 s(−17.3%) | 256.2 → 231.2 s | 6.58 → 6.18 GiB | +| 开发迭代:master `build`,本分支 `build:src` | 317.1 s | 153.6 s(−51.6%) | 423.1 → 231.2 s(−45.4%) | 6.11 → 6.18 GiB | +| `pnpm run build:test`(`build:src` 之后) | — | 67.7 s | 144.6 s | 5.33 GiB | +| `pnpm run build:test` 无改动重跑 | — | 19.4 s | 33.1 s | 未测量 | +| `lint` 与 `doc-typecheck` 的前置步骤:master `build:lib:host`,本分支 `host-libraries` | 159.8 s | 90.9 s(−43.2%) | 234.7 → 136.0 s | 未测量 | -本分支 `build` 的步骤 wall:native 1.7、host-src 66.5、host-bundle 11.4、desktop-bundle 3.4、client-src 45.6、client-bundle 9.0、web 9.4、host-test 65.3、client-test 34.8 s。按这些值计算,关键路径是 native → host-src → host-bundle → desktop-bundle → client-src → client-test,共 163.4 s;host-test 在第 133.5 s 结束,不在关键路径上。master 串行各段:native 2.5、Host 包编译 66.7、Host 聚合检查 66.1、Host tsdown 22.5、Desktop bundle 3.3、Client 包编译 55.6、Client 聚合检查 68.2、Client tsdown 13.9、web 9.7 s,合计 308.5 s。两者的差异也包含[无损提速 Note](2026-10-11-lossless-build-speedups.zh.md) 的改动:Host tsdown、Client 包编译、Client tsdown 与 Client 聚合检查的缩短来自该 Note,Host 聚合检查移出关键路径来自本 Note 的并发调度。`build:test` 的 67.9 s 中,Host 与 Client 聚合检查并发运行,分别为 65.2 s 与 33.9 s。 +本分支串行样本运行时主机上有其他工作(外来 CPU 742 s),因此它的 wall 可能高估串行成本。串行构建的各段中(master → 本分支,各一个样本),Host tsdown 从 22.5 s 降到 11.7 s,Client 包编译从 55.6 s 降到 48.7 s,Client 聚合检查从 68.2 s 降到 35.9 s,Client tsdown 从 13.9 s 降到 10.5 s;各段合计分别为 308.5 s 与 254.9 s。这些缩短来自[无损提速 Note](2026-10-11-lossless-build-speedups.zh.md);从 254.9 s 的串行各段合计到 169.1 s 的并发构建,其余缩短来自本 Note 的调度,它把 Host 聚合检查(约 66 s)移出关键路径。并发构建的关键路径是 native → host-src → host-bundle → desktop-bundle → client-src → client-test,host-test 在最后一步结束前约 30 s 完成(取自更早的一个样本)。`build:test` 并发运行 Host 与 Client 聚合检查,因此其 wall 等于 Host 聚合检查的 wall(约 65 s)。 ### 完整性与无损依据 @@ -119,14 +119,16 @@ wall 中位数从 249.4 s 降到 172.1 s(−77.3 s,−31%);CPU 增加约 负对照:在只属于 Host 聚合根程序的 `scripts/pnpm-invocation.spec.ts` 中加入类型错误(TS2322)后,`build:src` 以 0 退出;`build:test` 以 1 退出,其中 Host 聚合检查失败、Client 聚合检查通过;`build` 以 1 退出,9 个步骤中 8 个通过、1 个失败,且不写 client build record。在临时 fixture 中,被引用 project 的 tsconfig 含无效 `target`、JSON 语法错误或未知编译选项时,`tsc -p` 以 0 退出,`tsc -b` 分别报告 TS6046、TS5092 与 TS1012、TS5023 并以 1 退出;被引用 project 缺少 `composite` 时,两者都报告 TS6306。 +CI 证据:PR #6005 上,两个运行并发 `pnpm run build` 的必需 lane 均通过:`node 24 / snapshots and artifacts`(`check:ci:consumers`,运行于 `blacksmith-16vcpu-ubuntu-2404`)与 `windows node 24 / build & observational`(`check:ci:windows-blocking`,运行于 `blacksmith-16vcpu-windows-2025`);GitHub 托管的 4 vCPU benchmarks lane 不运行 `build`。内层并发取 `DSH_GATE_CONCURRENCY`:master 串行 lane 设为 1,consumers lane 在 self-hosted failover 时设为 10,未设置时取可用 CPU 数,且不超过 9 个步骤。 + ### 有意的取舍(非无损) - **部分命令跳过聚合检查:** `build:src`、`dev:web` 与 `dev:desktop` 的启动构建,以及 `lint`、`lint:fix`、`doc-typecheck` 与 Typert contracts gate 的前置步骤,都不类型检查两个聚合根程序,这些前置步骤也不产出 `apps/desktop/lib`。完整性由 `build`、`build:test`、`typecheck`(pre-push hook)与两个必需 CI lane 负责;`dev:web` 与 `dev:desktop` 不会因测试类型错误而拒绝启动。 - **`build:test` 不检查新鲜度:** 在 `build:src` 之后修改源码时,`build:test` 基于旧声明(包括 Typert Remote 声明)检查;需要先重跑 `build:src`。完整 `build` 不受影响。 -- **并发 `build` 提高内存峰值:** 本分支完整 `build` 的进程树 RSS 峰值为 7.55 GiB,master 串行构建为 6.13 GiB(+1.42 GiB,各一个样本);同时 CPU 从 425.5 s 降到 373.4 s,因为[无损提速 Note](2026-10-11-lossless-build-speedups.zh.md) 省去的工作多于并发争用增加的工作。原型(不含 Typert 改动)中,单独的并发调度使 RSS 峰值增加约 2.1 GiB(5.9 → 8.0 GiB)、CPU 增加约 3.7%。上限由 `DSH_GATE_CONCURRENCY` 控制;内存紧张时使用 `DSH_GATE_CONCURRENCY=1 pnpm run build`,或分别运行 `build:src` 与 `build:test`。windows-build lane 中 `build` 与 `docs:build` 同时运行。 +- **并发 `build` 提高内存峰值:** 本分支完整 `build` 的进程树 RSS 峰值为 7.53 GiB,master 串行构建为 6.11 GiB(+1.42 GiB,2 个样本的中位数),本分支串行构建为 6.27 GiB(n=1);同时 CPU 从 423.1 s 降到 374.4 s,因为[无损提速 Note](2026-10-11-lossless-build-speedups.zh.md) 省去的工作多于并发争用增加的工作。原型(不含 Typert 改动)中,单独的并发调度使 RSS 峰值增加约 2.1 GiB(5.9 → 8.0 GiB)、CPU 增加约 3.7%。上限由 `DSH_GATE_CONCURRENCY` 控制;内存紧张时使用 `DSH_GATE_CONCURRENCY=1 pnpm run build`,或分别运行 `build:src` 与 `build:test`。windows-build lane 中 `build` 与 `docs:build` 同时运行。 - **成功步骤的输出默认不显示:** 只有失败步骤打印完整输出。 -- **`tsc -p` 不跳过已是最新的根程序:** 无改动时立即重跑 `build:test` 仍需 19.4 s,因为两个程序都要重新解析;`tsc -b` 会跳过已是最新的根。 -- **多核冷启动时 `build:src` 只比 `build` 快 17.6 s:** 本分支冷启动 `build` 为 170.1 s,`build:src` 为 152.5 s(各一个样本),因为 host-test 不在关键路径上,client-test(34.8 s)只比 client-bundle 与 web 之和(18.4 s)长约 16 s。`build:src` 的其余收益是 CPU 与内存:原型中两个聚合检查合计约 148 s CPU,Host 聚合检查与 Host tsdown 重叠时使内存峰值增加约 2 GiB;在少核机器与增量重建中,这部分也转化为 wall。 +- **`tsc -p` 不跳过已是最新的根程序:** 无改动时立即重跑 `build:test` 仍需 19.4 s 与 33.1 s CPU,因为两个程序都要重新解析;`tsc -b` 会跳过已是最新的根。 +- **多核冷启动时 `build:src` 只比 `build` 快 15.5 s:** 本分支冷启动 `build` 为 169.1 s,`build:src` 为 153.6 s(2 个样本的中位数),因为 host-test 不在关键路径上,client-test 只比 client-bundle 与 web 之和长约 16 s。`build:src` 的其余收益是 CPU 与内存:CPU 从 374.4 s 降到 231.2 s,RSS 峰值从 7.53 GiB 降到 6.18 GiB;在少核机器与增量重建中,这部分也转化为 wall。 - **CI 调用根 manifest 中的公开命令:** PR 产物构建调用 `build:src`,而不是内部 helper 参数;修改根 manifest 中这些 script 的 PR 会激活独立的 Node Addon System 平台矩阵。 ## 考虑过的替代方案 @@ -144,7 +146,7 @@ wall 中位数从 249.4 s 降到 172.1 s(−77.3 s,−31%);CPU 增加约 ## 后果 -开发迭代可以用 `build:src` 跳过两个聚合根程序(冷启动比 master 的 `build` 快 52%),在声明完成前用 `build:test` 或 `build` 补齐检查。`build` 保持完整:Host 聚合检查不在关键路径上,Client 聚合检查与 Client bundle、web 并行,冷启动比 master 快 46.5%(含无损提速 Note 的改动,各一个样本)。代价是更高的内存峰值、并发争用带来的少量额外 CPU、默认更少的成功输出,以及多个开发与前置命令不检查测试类型(见上文取舍)。 +开发迭代可以用 `build:src` 跳过两个聚合根程序(冷启动比 master 的 `build` 快 51.6%),在声明完成前用 `build:test` 或 `build` 补齐检查。`build` 保持完整:Host 聚合检查不在关键路径上,Client 聚合检查与 Client bundle、web 并行,冷启动比 master 快 46.7%(含无损提速 Note 的改动,2 个样本的中位数)。代价是更高的内存峰值、并发争用带来的少量额外 CPU、默认更少的成功输出,以及多个开发与前置命令不检查测试类型(见上文取舍)。 删除或改变 consumers 与 windows-build 两个必需 lane 的 `build`,必须先把两个聚合检查转交另一处阻塞型 owner。若 Typert 以后获得独立的生成步骤,步骤表新增该步骤,client-src 改为依赖它;命令语义与调度不变。 diff --git a/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.i18n.yaml b/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.i18n.yaml index b4e9d1690d..58eda46b99 100644 --- a/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.i18n.yaml +++ b/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.i18n.yaml @@ -6,41 +6,41 @@ en: 48682b646b8f993a zh: 4c26cb4aab5cd08c /agent-note-build-speed-ups-that-keep-outputs-unchanged/problem: - en: 874d7d87e3d2a7c6 - zh: 06bdfd1eb91c25e4 + en: 46551eb7e33c40ca + zh: f8c1fc063fa77062 /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision: - en: 59f288bb758dde74 - zh: d2360c02701cf96c + en: 97bc6376ec8ebd2e + zh: 25bf2941ddb5e26c /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision/measurement-conditions: en: 2ab844073ec7c336 zh: b25e39f2f80b0d7e /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision/1-only-the-root-manifest-declares-react-type-packages: - en: 1cd139ef5c258dfe - zh: 9397dc3b8530b9ae + en: 1101c94d04f9c7e9 + zh: 413b7b15f98a5f62 /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision/2-typert-selects-packages-by-manifest-first: - en: 824472b4248bbb9d - zh: 411479c8bc830b06 + en: 1ae4b65792db0975 + zh: a11c0d90842a3b55 /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision/3-typert-does-not-expand-aggregate-include: - en: 6c2348fd12771991 - zh: a2b7ea95f471fe20 + en: 0c8ab2d412113914 + zh: 01ba4f5042422d77 /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision/4-typert-parses-package-tsconfigs-lazily: - en: 1d6876419e852e68 - zh: d526c7dc0f94118d + en: cd6e05c61a7cc495 + zh: 480d2d85d78d2d7f /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision/5-the-typert-workspace-root-also-requires-packages: - en: 7bec99d36d8d3fd0 - zh: ee4273df6178180a + en: 8b89436df201141c + zh: 2fab82419c8ba704 /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision/6-typert-generates-in-a-worker-thread: - en: 0f2450529c1b1b80 - zh: c6ed12cb882131e6 + en: 0362483a79c07881 + zh: d84a2390b49c4012 /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision/7-the-typert-transform-hook-filters-by-module-id: - en: e738cc16329da590 - zh: 7d04be9a9f5837d7 + en: b5ad6b5506c150d5 + zh: 25f09e4ab74696ad /agent-note-build-speed-ups-that-keep-outputs-unchanged/decision/8-client-tsdown-presets-read-the-manifest-from-the-package-directory: - en: 388f070a74bcc18f - zh: 23b14fcb96007a6c + en: 80c2ddc5fdff0b5f + zh: f5ce309a2e88ff4f /agent-note-build-speed-ups-that-keep-outputs-unchanged/alternatives-considered: en: 186e42f2f5481c8d zh: 4a1c79a391a268b0 /agent-note-build-speed-ups-that-keep-outputs-unchanged/consequences: - en: 5712ab878322479b - zh: f5acdcbffc819bf2 + en: aea05af999315af1 + zh: a8e6fca3be2306f9 diff --git a/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.md b/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.md index 2706b70bb8..53673cb83f 100644 --- a/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.md +++ b/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.md @@ -8,7 +8,7 @@ English | [中文](2026-10-11-lossless-build-speedups.zh.md) A cold build carries three fixed costs that do not affect output contents: -- **JSX signatures in the Client test aggregate program.** The Check phase of `tsc -p tsconfig.client.json` takes 59.1 s, more than half of it computing the `@types/react` module specifier for every JSX file that imports React. TypeScript's `computeModuleSpecifiers` visits every symlink location of that declaration file and scans all 491 tsconfig `paths` entries at each location. pnpm creates one `node_modules/@types/react` link for each workspace package that declares `@types/react` itself; 66 manifests declare it, so the Client test aggregate program sees 65 locations. Counters measured 88 JSX files × 2 modules (`index.d.ts`, `jsx-runtime.d.ts`) × 65 locations: 11,457 `tryGetModuleNameFromPaths` calls and 5,623,782 entry scans, 32.6 s in total. A dose-response run that truncates the visible location count N (N = 1, 16, 32, 48, 65; Check 26.92, 34.03, 44.92, 50.17, 58.40 s) gives Check ≈ 26.9 + 0.494·N s. +- **JSX signatures in the Client test aggregate program.** The Check phase of `tsc -p tsconfig.client.json` takes 59.1 s, more than half of it computing the `@types/react` module specifier for every JSX file that imports React. TypeScript's `computeModuleSpecifiers` visits every symlink location of that declaration file and scans all 491 tsconfig `paths` entries at each location. pnpm creates one `node_modules/@types/react` link for each workspace package that declares `@types/react` itself; 66 manifests declare it, so the Client test aggregate program sees 65 locations. Counters measured 88 JSX files × 2 modules (`index.d.ts`, `jsx-runtime.d.ts`) × 65 locations: 11,457 `tryGetModuleNameFromPaths` calls and 5,623,782 entry scans, 32.6 s in total. A dose-response run at five truncation levels of the visible location count N (N = 1 to 65; Check 26.92 s to 58.40 s) fits Check ≈ 26.9 + 0.494·N s. - **Typert inside the Host tsdown pass.** The Host tsdown pass takes about 21.6–22.6 s. Typert discovers and generates artifacts synchronously inside the first `writeBundle`; the CPU profile shows it blocking the tsdown main thread for about 12.4 s while every package build waits. Discovery takes 8.06 s: it parses all 362 package tsconfigs referenced by the two face aggregates (about 3.8 s), expands the Host aggregate's 2,064 literal `include` entries on every parse (1.6 s; the analyzer never reads those file names), and walks the public-export surface of 322 registrations (2.7–3.2 s). Only the 28 packages that declare a Typert export produce Host artifacts. - **Manifest lookup in the Client tsdown pass.** The Client tsdown pass takes about 11.7 s with a saturated main thread (7.1% idle); 34.5% of it (3.95 s) is `workspaceManifest`: every package config runs `globSync('packages/*/*/package.json')` for its package name and parses manifests until the name matches. The module-level cache has no effect because tsdown's native config loader imports every package config through `import-without-cache`, creating a fresh module instance per config. Each glob takes 40–125 ms and runs inside the `neverBundle`/`alwaysBundle` callbacks of `tsdown:deps`, blocking every build. @@ -29,6 +29,8 @@ The following eight speed-ups change neither the bytes of `lib/` and `apps/web/d | 7 | The Typert `transform` hook filters by module id | Host tsdown | Within noise; hook calls 1,566 → 195 | Byte-identical | None | | 8 | Client tsdown presets read the manifest from the package directory | Client tsdown | 12.78 → 8.73 s | Byte-identical; external decisions identical | The presets' first argument is the package directory | +The published `./tsdown` export `typertPlugin()` changes with items 5–7 (`transform` as `{ filter, handler }`, an asynchronous `writeBundle`, and a workspace root that must contain `packages/`) without needing an upgrade guide, because rolldown and tsdown consumers accept both hook forms and the analyzer already required `/packages`, so no external workspace that worked before stops working. + ### Measurement conditions All numbers come from one shared Linux host (256 vCPU, 4 TB memory, load average 23–111 during runs) with Node v24.18.0, TypeScript 6.0.3, and tsdown 0.22.2 (rolldown 1.1.1). @@ -46,11 +48,11 @@ All numbers come from one shared Linux host (256 vCPU, 4 TB memory, load average | Measurement | Per-package declarations (N = 65) | Root manifest declaration (N = 1) | |---|---|---| - | Client test aggregate Check | 59.08 s (58.76 / 59.08 / 59.39) | 26.28 s (5 samples, 26.04–26.60), −56% | + | Client test aggregate Check | 59.08 s (median of 3) | 26.28 s (median of 5), −56% | | Client test aggregate wall | 68–71 s | 33.6–34.9 s | | Client test aggregate user CPU | 79–83 s | 44.5–45.3 s | - | Check sum of the 76 client-only package projects | 34.48 / 34.65 s | 26.47 / 26.61 s (−23%) | - | Wall of the 76 client-only package projects | 54.0 / 57.4 s | 44.8 / 44.2 s | + | Check sum of the 76 client-only package projects | 34.6 s (median of 2) | 26.5 s (median of 2), −23% | + | Wall of the 76 client-only package projects | 55.7 s (median of 2) | 44.5 s (median of 2) | - **Why it is lossless:** The 66 package-level links and the root link share one realpath (`.pnpm/@types+react@18.3.31`), so the program reads the same declarations; the program keeps 2,613 files, and `tsc -b --force tsconfig.client.json` exits 0 without diagnostics. `devDependencies` are not installed with published packages, so the [published dependency faces](2026-08-26-published-dependency-faces.md) conclusion that independent TypeScript consumers install referenced type packages themselves is unchanged; runtime resolution never reads `@types/*`. - **Trade-off:** Package manifests do not list their own React type dependency, and every workspace package shares one React types version; a package that needs a different version requires a new Agent Note overturning this item. The check covers only `apps/*`, `packages/*/*`, and `vendor/*`; other pnpm members such as `website`, `benchmarks`, `native/system`, and `python/sdk-runtime` are out of scope, and none of them currently declares React type packages. @@ -59,21 +61,21 @@ All numbers come from one shared Linux host (256 vCPU, 4 TB memory, load average - **Purpose:** Workspace mode generates artifacts only for packages that declare a Typert export, so it skips the surface walk for every other candidate. - **Change:** `WorkspaceAnalyzer.discoverPackages(includeManifest?)` and `WorkspaceTypertGenerator.discover(faces?, includeManifest?)` accept an optional `package.json` predicate applied before the surface walk; a rejected package has neither its sources read nor its project config parsed. `WorkspaceTypertGenerator.generateContributors(faces?)` passes `hasTypertExport` (`exports` contains `./typert`, `./client/typert`, or `./remote`). Callers that pass no predicate (the Cordis catalog, `generate()` without packages) behave as before. -- **Performance:** Host tsdown 22.62 s (21.81 / 22.62 / 23.40) → 19.94 s (19.26 / 19.94 / 20.37). +- **Performance:** Host tsdown 22.62 s → 19.94 s (medians of 3). - **Why it is lossless:** The predicate is the filter previously applied after discovery, and a package it rejects never produced artifacts; the 743 output files are byte-identical except the Typert generator's own `lib/index.js`, whose source is this change. ### 3. Typert does not expand aggregate `include` - **Purpose:** The analyzer reads only an aggregate's compiler options and project references; expanding `include` only fills `fileNames`, which it never uses. - **Change:** `WorkspaceCaches.aggregate(path)` returns and caches `AggregateConfig { path, options, projectReferences }`. An aggregate that declares its own `references` is parsed with a `ParseConfigHost` whose `readDirectory` returns an empty list; an aggregate without `references` is expanded as usual. -- **Performance:** Host tsdown 19.94 s → 16.99 s (16.97 / 16.99 / 19.08). +- **Performance:** Host tsdown 19.94 s → 16.99 s (medians of 3). - **Why it is lossless:** TypeScript reports TS18003 (no inputs) only for a config with neither `files` nor `references`, so an aggregate that declares `references` yields the same diagnostics, options, and references without expansion. The registration inventory dumped after forcing every lazy config (408 registrations, including `fileNames` and options, plus the aggregates' options and references) is byte-identical to the baseline, and the outputs are byte-identical. The tests `reads aggregate options and references without expanding include patterns` and `reports an aggregate without references or inputs` pin both kinds of config. ### 4. Typert parses package tsconfigs lazily - **Purpose:** Parse only the tsconfigs of packages selected for analysis, not every package that an aggregate references. - **Change:** `PackageRegistration.config` is a getter over `caches.config(configPath)` that parses on first read. The registration inventory keeps its order, faces, names, roots, and export subpaths. -- **Performance:** Host tsdown 16.99 s → 16.25 s (16.25 / 14.72 / 16.38), within the group spread. Together, items 2–4 reduce Typert discovery in an isolated benchmark from 5.9–7.1 s to 0.36–0.40 s, tsconfig parses from 362 to 28, surface walks from 322 to 28, and total Typert time from 9.7–11.8 s to 4.4–4.5 s. +- **Performance:** Host tsdown 16.99 s → 16.25 s (medians of 3), within the 2.5 s within-group spread. Together, items 2–4 reduce Typert discovery in an isolated benchmark from 5.9–7.1 s to 0.36–0.40 s, tsconfig parses from 362 to 28, surface walks from 322 to 28, and total Typert time from 9.7–11.8 s to 4.4–4.5 s. - **Why it is lossless:** The inventory comparison is the one in item 3 (byte-identical), and the outputs are byte-identical. The test `reads only the project configs of manifests selected for discovery` pins that only selected configs are parsed. - **Trade-off (not lossless):** Typert does not parse tsconfigs of packages it does not analyze, so a syntax or configuration error in such a tsconfig does not fail Typert. In the same build, the Host package compilation `tsc -b` that runs before the Host bundle and the Client package compilation `tsc -b` parse every referenced tsconfig and reject those errors; the `PackageRegistration.config` JSDoc and the [Typert generator README](../../../../packages/typert/generator/README.md) state this ownership. A consumer that runs Typert without `tsc -b` does not receive those diagnostics. @@ -81,7 +83,7 @@ All numbers come from one shared Linux host (256 vCPU, 4 TB memory, load average - **Purpose:** Correct workspace-root resolution. 24 dual-face packages carry their own `tsconfig.host.json` leaf; searching upward for `tsconfig.host.json` alone took each of them as a root, so workspace mode ran one empty discovery per such package per build, and `package` mode silently produced no artifacts for dual-face packages. - **Change:** The workspace root is the nearest ancestor of the bundle output that contains both `tsconfig.host.json` and `packages/`. -- **Performance:** Host tsdown 14.82 s → 15.39 s (15.39 / 15.47 / 14.16), within noise. +- **Performance:** Host tsdown 14.82 s → 15.39 s (medians of 3), within noise. - **Why it is lossless:** The repository root [`tsdown.config.ts`](../../../../tsdown.config.ts) uses only workspace mode, and the 743 output files are byte-identical. The test `resolves the workspace root above a dual-face package leaf` pins the rule. - **Behavior difference:** A `package`-mode consumer building a dual-face package receives that package's Typert artifacts; repository builds do not use `package` mode. @@ -89,7 +91,7 @@ All numbers come from one shared Linux host (256 vCPU, 4 TB memory, load average - **Purpose:** Overlap Typert analysis with bundling on the tsdown main thread instead of blocking every package build inside the first `writeBundle`. - **Change:** [`src/worker.ts`](../../../../packages/typert/generator/src/worker.ts) is the Worker Thread entry; it runs `generateContributors(faces)` and posts one `artifacts` or `failure` message. In workspace mode, the build's first `buildStart` locates the workspace root from `options.cwd` and starts generation; `writeBundle` is asynchronous, the first finishing bundle claims emission and awaits the result, and later bundles return immediately. An analysis error, a worker crash or early exit, a malformed or undeserializable response, or a worker that cannot be terminated fails the claiming bundle with a `TypertWorkerError` naming the workspace and cause, which fails the whole tsdown pass; the worker is terminated before the outcome settles, and an unclaimed outcome never becomes an unhandled rejection. Built output starts the worker from `./worker.js`; source runs (tests) load `worker.ts` through `tsx/esm/api`. `package` mode still generates on the main thread. -- **Performance:** Host tsdown 15.39 s → 10.25 s (11.31 / 10.07 / 10.25). Profile of the final configuration (items 2–7): main thread 10.1 s, first `buildStart` at 4.18 s (tsdown config resolution), `writeBundle` on the main thread only 0.37 s; worker 5.0 s (module load about 0.4 s, discovery 0.28 s, generation 3.88 s including `createProgram` 2.21 s), ending at about 9.3 s. Host tsdown wall is about max(bundling, 4.2 s + Typert). +- **Performance:** Host tsdown 15.39 s → 10.25 s (medians of 3). Profile of the final configuration (items 2–7): main thread 10.1 s, first `buildStart` at 4.18 s (tsdown config resolution), `writeBundle` on the main thread only 0.37 s; worker 5.0 s (module load about 0.4 s, discovery 0.28 s, generation 3.88 s including `createProgram` 2.21 s), ending at about 9.3 s. Host tsdown wall is about max(bundling, 4.2 s + Typert). - **Why it is lossless:** All 54 second-round runs exit 0, and each run's 743 output files match the baseline except the generator's own `lib/index.js`. The test `writes the artifacts in-process generation produces` compares worker and in-process artifacts; in a scratch workspace with one package's `files` broken, tsdown exits 1 within 2.4 s, reporting `TypertWorkerError` with the worker's `TypertAnalysisError`, and writes no artifacts. The `typertPlugin workspace mode` test group covers each failure path. - **Trade-off:** The worker heap is about 300–420 MiB; the final configuration's median Host tsdown max RSS is 6,296–6,408 MiB against 5,676 MiB at baseline. Under heavy shared-host load the worker can land on the critical path: one sample took 21.91 s with the artifact-owning package build waiting about 14 s for the worker, not reproduced in 10 later samples; at load average about 100, two runs took 17.86 s and 14.3 s. The worker also completes under a 1,024 MiB heap limit, so GC pressure is not the cause; attribution to host contention is unproven. @@ -97,14 +99,14 @@ All numbers come from one shared Linux host (256 vCPU, 4 TB memory, load average - **Purpose:** Stop every module from crossing from rolldown into the JavaScript `transform` hook only to be rejected by an extension check. - **Change:** `transform` declares `filter: { id: /^[^?]*\.[cm]?tsx?(?:\?|$)/ }`, and its handler still checks the id with the same regular expression. -- **Performance:** Hook calls per Host build fall from 1,566 to 195 (195 TypeScript modules, 4 of them with decorators). Wall is 11.07 s (11.07 / 10.21 / 11.30) and 9.98 s (4 samples) against 10.25 s for item 6, while the identical-code control groups give 11.07 s and 9.87 s; the difference is within noise. +- **Performance:** Hook calls per Host build fall from 1,566 to 195 (195 TypeScript modules, 4 of them with decorators). Wall medians are 11.07 s (3 samples) and 9.98 s (4 samples) against 10.25 s for item 6, while the identical-code control groups give 11.07 s and 9.87 s; the difference is within noise. - **Why it is lossless:** The regular expression uses only ASCII constructs and matches the same id set in rolldown's Rust regex engine and in JavaScript; the handler applies the same check, so hosts that ignore filters behave the same. Outputs are byte-identical; the test `filters transform calls to TypeScript module ids` pins the filter. ### 8. Client tsdown presets read the manifest from the package directory - **Purpose:** Each package config reads only its own `package.json`, removing the glob over every manifest by package name. - **Change:** The first argument of `clientBundle`, `staticLinked`, and `clientLibrary` in [`packages/client/tsdown.client.ts`](../../../../packages/client/tsdown.client.ts) is the package directory, normally the config file's `import.meta.dirname`; the preset reads `/package.json` once and uses its `name` as the plugin id and diagnostic name; when that manifest is missing, unreadable, or has no `name`, config loading fails with an error naming the path and cause. 82 package configs call these presets. The presets keep no module-level manifest cache; the Node half's production-dependency external list is computed from that manifest on first resolve and cached in that config's closure. -- **Performance:** Client tsdown falls from 12.78 s (13.77 / 12.78 / 12.31) to 8.73 s (8.19 / 8.80 / 8.73), −4.05 s (−31.7%), with disjoint sample ranges; user CPU falls from 26.1 s to 25.2 s and sys from 5.9 s to 4.9 s, so the saving comes mainly from unblocking the main thread rather than from less total CPU. The prototype (one glob building a name index) measured 11.72 s → 7.46 s (−36%). In Host tsdown only 5 `hostPhase` packages reach the lookup; one sample pair measured 11.92 s → 10.71 s, with low confidence. +- **Performance:** Client tsdown falls from 12.78 s to 8.73 s (medians of 3), −4.05 s (−31.7%), with disjoint sample ranges; user CPU falls from 26.1 s to 25.2 s and sys from 5.9 s to 4.9 s, so the saving comes mainly from unblocking the main thread rather than from less total CPU. The prototype (one glob building a name index) measured 11.72 s → 7.46 s (−36%). In Host tsdown only 5 `hostPhase` packages reach the lookup; one sample pair measured 11.92 s → 10.71 s, with low confidence. - **Why it is lossless:** The 11,566 lib files are byte-identical across 11 runs. After the 82 call sites switched to package directories, no resolved package name differs from the name previously written at the call site. An in-process comparison of the external decisions for 82 configs × 3 faces × 1,600 specifiers matches the original implementation exactly; a negative control with one extra external changes 75 decisions. `client-bundle-purity.spec.ts` covers these manifest errors. - **Behavior difference:** The presets' calling convention changes from a package name to a package directory. @@ -125,7 +127,7 @@ All numbers come from one shared Linux host (256 vCPU, 4 TB memory, load average ## Consequences -Client type checking spends substantially less CPU and wall time, and the Host and Client tsdown passes take substantially less wall time, without changing runtime resolution, published installation semantics, or any output byte. In one complete `build`, Host tsdown takes 11.4 s, Client tsdown 9.0 s, and the Client aggregate check 34.8 s, against 22.5, 13.9, and 68.2 s in the master serial build (one sample each; see [the build split note](2026-10-11-build-src-test-split.md)). Host tsdown lies on the critical path of both `build` and `build:src`, Client tsdown on that of `build:src`, and the Client aggregate check is the last step of the `build` critical path. +Client type checking spends substantially less CPU and wall time, and the Host and Client tsdown passes take substantially less wall time, without changing runtime resolution, published installation semantics, or any output byte. In the serial build segments (master → branch, one sample each), Host tsdown falls from 22.5 to 11.7 s, Client package compilation from 55.6 to 48.7 s, the Client aggregate check from 68.2 to 35.9 s, and Client tsdown from 13.9 to 10.5 s; the serial segments sum to 308.5 s on master and 254.9 s on the branch (see [the build split note](2026-10-11-build-src-test-split.md)). Host tsdown lies on the critical path of both `build` and `build:src`, Client tsdown on that of `build:src`, and the Client aggregate check is the last step of the `build` critical path. Costs: every workspace package shares one React types version; Typert does not validate tsconfigs of packages it does not analyze, and `tsc -b` owns those errors; Host tsdown max RSS grows by about 0.6 GiB, and under heavy shared-host load the worker can become Host tsdown's critical path. diff --git a/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.zh.md b/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.zh.md index b00ff55621..59ef6b6618 100644 --- a/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.zh.md +++ b/.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.zh.md @@ -8,7 +8,7 @@ Status: implemented 冷启动构建中有三处与产物内容无关的固定开销: -- **Client 测试聚合程序的 JSX 签名。** `tsc -p tsconfig.client.json` 的 Check 阶段为 59.1 s,其中一半以上用于为每个导入 React 的 JSX 文件计算 `@types/react` 的 module specifier。TypeScript 的 `computeModuleSpecifiers` 遍历该声明文件的每个符号链接位置,并在每个位置扫描全部 491 条 tsconfig `paths`。pnpm 为每个自行声明 `@types/react` 的 workspace 包创建一个 `node_modules/@types/react` 链接;66 个 manifest 声明了它,Client 测试聚合程序因此看到 65 个位置。计数器测得 88 个 JSX 文件 × 2 个模块(`index.d.ts`、`jsx-runtime.d.ts`)× 65 个位置,共 11,457 次 `tryGetModuleNameFromPaths` 调用、5,623,782 次条目扫描,合计 32.6 s。按可见位置数 N 截断的剂量实验(N = 1、16、32、48、65,Check 依次为 26.92、34.03、44.92、50.17、58.40 s)给出 Check ≈ 26.9 + 0.494·N s。 +- **Client 测试聚合程序的 JSX 签名。** `tsc -p tsconfig.client.json` 的 Check 阶段为 59.1 s,其中一半以上用于为每个导入 React 的 JSX 文件计算 `@types/react` 的 module specifier。TypeScript 的 `computeModuleSpecifiers` 遍历该声明文件的每个符号链接位置,并在每个位置扫描全部 491 条 tsconfig `paths`。pnpm 为每个自行声明 `@types/react` 的 workspace 包创建一个 `node_modules/@types/react` 链接;66 个 manifest 声明了它,Client 测试聚合程序因此看到 65 个位置。计数器测得 88 个 JSX 文件 × 2 个模块(`index.d.ts`、`jsx-runtime.d.ts`)× 65 个位置,共 11,457 次 `tryGetModuleNameFromPaths` 调用、5,623,782 次条目扫描,合计 32.6 s。按可见位置数 N 截断的剂量实验在五个截断级别(N 从 1 到 65,Check 从 26.92 s 到 58.40 s)上拟合出 Check ≈ 26.9 + 0.494·N s。 - **Host tsdown 阶段中的 Typert。** Host tsdown 阶段约 21.6–22.6 s。Typert 在第一个 `writeBundle` 中同步发现并生成产物,CPU profile 显示它阻塞 tsdown 主线程约 12.4 s,期间所有包构建都在等待。发现阶段占 8.06 s:它解析两个 face 聚合配置所引用的 362 个包 tsconfig(约 3.8 s),每次解析 Host 聚合配置都展开 2,064 条字面 `include`(1.6 s,分析器从不读取这些文件名),并对 322 个注册项做公开导出表面遍历(2.7–3.2 s);最终只有 28 个声明 Typert 导出的包产生 Host 产物。 - **Client tsdown 阶段的 manifest 查找。** Client tsdown 阶段约 11.7 s,主线程饱和(空闲仅 7.1%),其中 34.5%(3.95 s)在 `workspaceManifest`:每个包配置按包名 `globSync('packages/*/*/package.json')` 并逐个解析 manifest,直到名字匹配。模块级缓存无效,因为 tsdown 的 native 配置加载器通过 `import-without-cache` 为每个包配置创建新的模块实例;每次 glob 耗时 40–125 ms,且发生在 `tsdown:deps` 的 `neverBundle`/`alwaysBundle` 回调中,阻塞全部构建。 @@ -29,6 +29,8 @@ Status: implemented | 7 | Typert `transform` 钩子按模块 id 过滤 | Host tsdown | 噪声内;钩子调用 1,566 → 195 | 逐字节一致 | 无 | | 8 | Client tsdown 预设按包目录读取 manifest | Client tsdown | 12.78 → 8.73 s | 逐字节一致;external 判定一致 | 预设第一个参数是包目录 | +第 5–7 项改变了发布的 `./tsdown` 导出 `typertPlugin()`(`transform` 为 `{ filter, handler }`、`writeBundle` 为异步、工作区根目录必须包含 `packages/`),但不需要升级指南:rolldown 与 tsdown 的使用方接受两种钩子形式,分析器本来就要求 `/packages`,因此此前可用的外部工作区都不会失效。 + ### 测量条件 所有数字取自同一台共享 Linux 主机(256 vCPU、4 TB 内存,运行期间 load average 23–111),Node v24.18.0、TypeScript 6.0.3、tsdown 0.22.2(rolldown 1.1.1)。 @@ -46,11 +48,11 @@ Status: implemented | 测量项 | 各包自声明(N = 65) | 根 manifest 声明(N = 1) | |---|---|---| - | Client 测试聚合 Check | 59.08 s(58.76 / 59.08 / 59.39) | 26.28 s(5 个样本 26.04–26.60),−56% | + | Client 测试聚合 Check | 59.08 s(3 个样本的中位数) | 26.28 s(5 个样本的中位数),−56% | | Client 测试聚合 wall | 68–71 s | 33.6–34.9 s | | Client 测试聚合 user CPU | 79–83 s | 44.5–45.3 s | - | 76 个 client-only 包项目的 Check 总和 | 34.48 / 34.65 s | 26.47 / 26.61 s(−23%) | - | 76 个 client-only 包项目的 wall | 54.0 / 57.4 s | 44.8 / 44.2 s | + | 76 个 client-only 包项目的 Check 总和 | 34.6 s(2 个样本的中位数) | 26.5 s(2 个样本的中位数),−23% | + | 76 个 client-only 包项目的 wall | 55.7 s(2 个样本的中位数) | 44.5 s(2 个样本的中位数) | - **无损依据:** 66 个包级链接与根链接指向同一个 realpath(`.pnpm/@types+react@18.3.31`),因此程序读取的声明内容相同;程序文件数保持 2,613,`tsc -b --force tsconfig.client.json` 以 0 退出且无诊断。`devDependencies` 不随已发布的包安装,[发布依赖面](2026-08-26-published-dependency-faces.zh.md)中独立 TypeScript 消费者自行安装所引用类型包的结论不变;runtime 解析不读取 `@types/*`。 - **取舍:** 包 manifest 不列出自己的 React 类型依赖,所有 workspace 包共用同一个 React 类型版本;某个包需要不同版本时,需要新的 Agent Note 推翻本项。检查只覆盖 `apps/*`、`packages/*/*` 与 `vendor/*`;`website`、`benchmarks`、`native/system` 与 `python/sdk-runtime` 等其他 pnpm 成员不在范围内,目前它们都不声明 React 类型包。 @@ -59,21 +61,21 @@ Status: implemented - **目的:** workspace 模式只为声明 Typert 导出的包生成产物,避免对其余候选包做表面遍历。 - **改动:** `WorkspaceAnalyzer.discoverPackages(includeManifest?)` 与 `WorkspaceTypertGenerator.discover(faces?, includeManifest?)` 接受可选的 `package.json` 谓词,在表面遍历之前应用;被拒绝的包既不读取源码,也不解析项目配置。`WorkspaceTypertGenerator.generateContributors(faces?)` 以 `hasTypertExport`(`exports` 含 `./typert`、`./client/typert` 或 `./remote`)为谓词。不传谓词的调用方(Cordis 目录、不指定包的 `generate()`)行为不变。 -- **性能:** Host tsdown 22.62 s(21.81 / 22.62 / 23.40)→ 19.94 s(19.26 / 19.94 / 20.37)。 +- **性能:** Host tsdown 22.62 s → 19.94 s(各 3 个样本的中位数)。 - **无损依据:** 谓词就是原先在发现之后应用的同一过滤条件,被它拒绝的包本来就不产生产物;743 个产物文件逐字节一致(Typert generator 自身的 `lib/index.js` 除外,其源码即本项改动)。 ### 3. Typert 不展开聚合配置的 `include` - **目的:** 分析器只读取聚合配置的编译选项与 project references,展开 `include` 只为填充从不使用的 `fileNames`。 - **改动:** `WorkspaceCaches.aggregate(path)` 返回并缓存 `AggregateConfig { path, options, projectReferences }`。解析自身声明 `references` 的聚合配置时,使用 `readDirectory` 返回空列表的 `ParseConfigHost`;不声明 `references` 的聚合配置照常展开。 -- **性能:** Host tsdown 19.94 s → 16.99 s(16.97 / 16.99 / 19.08)。 +- **性能:** Host tsdown 19.94 s → 16.99 s(各 3 个样本的中位数)。 - **无损依据:** TypeScript 只对既无 `files` 也无 `references` 的配置报告 TS18003(无输入),因此声明 `references` 的聚合配置在不展开时得到相同的诊断、选项与 references。强制读取每个惰性配置后导出的注册清单(408 个注册项,含 `fileNames` 与选项,以及聚合配置的选项和 references)与基线逐字节一致;产物逐字节一致。测试 `reads aggregate options and references without expanding include patterns` 与 `reports an aggregate without references or inputs` 固定这两种配置的行为。 ### 4. Typert 惰性解析包 tsconfig - **目的:** 只解析被选中分析的包的 tsconfig,而不是聚合配置引用的全部包。 - **改动:** `PackageRegistration.config` 是 `caches.config(configPath)` 的 getter,首次读取时解析。注册清单的顺序、face、包名、根目录与导出子路径不变。 -- **性能:** Host tsdown 16.99 s → 16.25 s(16.25 / 14.72 / 16.38),在组内跨度以内。第 2–4 项合计在隔离基准中把 Typert 发现从 5.9–7.1 s 降到 0.36–0.40 s,tsconfig 解析从 362 次降到 28 次,表面遍历从 322 次降到 28 次,Typert 总耗时从 9.7–11.8 s 降到 4.4–4.5 s。 +- **性能:** Host tsdown 16.99 s → 16.25 s(各 3 个样本的中位数),在 2.5 s 的组内跨度以内。第 2–4 项合计在隔离基准中把 Typert 发现从 5.9–7.1 s 降到 0.36–0.40 s,tsconfig 解析从 362 次降到 28 次,表面遍历从 322 次降到 28 次,Typert 总耗时从 9.7–11.8 s 降到 4.4–4.5 s。 - **无损依据:** 注册清单比较与第 3 项相同(逐字节一致),产物逐字节一致。测试 `reads only the project configs of manifests selected for discovery` 固定只解析所选包的配置。 - **取舍(非无损):** Typert 不解析它不分析的包的 tsconfig,因此这类 tsconfig 的语法或配置错误不会使 Typert 失败。同一构建中,先于 Host bundle 运行的 Host 包编译 `tsc -b` 与 Client 包编译 `tsc -b` 解析每个被引用的 tsconfig,并拒绝其中的错误;`PackageRegistration.config` 的 JSDoc 与 [Typert generator README](../../../../packages/typert/generator/README.zh.md) 写明这一归属。单独调用 Typert 而不运行 `tsc -b` 的使用方不会得到这些诊断。 @@ -81,7 +83,7 @@ Status: implemented - **目的:** 修正工作区根目录的判定。24 个双 face 包带有自己的 `tsconfig.host.json` 叶子配置;只按 `tsconfig.host.json` 向上查找时,每个这样的包都被当作根目录,workspace 模式每次构建为它们各做一次空发现,`package` 模式下双 face 包得不到产物且没有报错。 - **改动:** 工作区根目录是打包输出最近的、同时包含 `tsconfig.host.json` 与 `packages/` 的祖先目录。 -- **性能:** Host tsdown 14.82 s → 15.39 s(15.39 / 15.47 / 14.16),在噪声内。 +- **性能:** Host tsdown 14.82 s → 15.39 s(各 3 个样本的中位数),在噪声内。 - **无损依据:** 仓库根 [`tsdown.config.ts`](../../../../tsdown.config.ts) 只使用 workspace 模式,743 个产物文件逐字节一致。测试 `resolves the workspace root above a dual-face package leaf` 固定该判定。 - **行为差异:** `package` 模式的使用方构建双 face 包时得到该包的 Typert 产物;仓库构建不使用 `package` 模式。 @@ -89,7 +91,7 @@ Status: implemented - **目的:** 让 Typert 分析与 tsdown 主线程上的打包重叠,而不是在第一个 `writeBundle` 中阻塞全部包构建。 - **改动:** [`src/worker.ts`](../../../../packages/typert/generator/src/worker.ts) 是 Worker Thread 入口,运行 `generateContributors(faces)` 并回传一条 `artifacts` 或 `failure` 消息。workspace 模式下,本次构建的第一个 `buildStart` 以 `options.cwd` 定位工作区根目录并启动生成;`writeBundle` 是异步的,第一个完成的打包认领写入并等待结果,之后的打包立即返回。分析错误、worker 崩溃或提前退出、格式错误或无法反序列化的响应、worker 无法终止,都以指明工作区与原因的 `TypertWorkerError` 使认领写入的打包失败,从而使整个 tsdown 阶段失败;结果敲定前 worker 已终止,未被认领的结果不会成为未处理的 rejection。构建产物从 `./worker.js` 启动 worker;源码运行(测试)通过 `tsx/esm/api` 加载 `worker.ts`。`package` 模式仍在主线程中生成。 -- **性能:** Host tsdown 15.39 s → 10.25 s(11.31 / 10.07 / 10.25)。最终配置(第 2–7 项)的 profile:主线程 10.1 s,第一个 `buildStart` 出现在 4.18 s(tsdown 配置解析),主线程上的 `writeBundle` 只占 0.37 s;worker 5.0 s(模块加载约 0.4 s、发现 0.28 s、生成 3.88 s,其中 `createProgram` 2.21 s),约在 9.3 s 结束。Host tsdown 的 wall 约为 max(打包, 4.2 s + Typert)。 +- **性能:** Host tsdown 15.39 s → 10.25 s(各 3 个样本的中位数)。最终配置(第 2–7 项)的 profile:主线程 10.1 s,第一个 `buildStart` 出现在 4.18 s(tsdown 配置解析),主线程上的 `writeBundle` 只占 0.37 s;worker 5.0 s(模块加载约 0.4 s、发现 0.28 s、生成 3.88 s,其中 `createProgram` 2.21 s),约在 9.3 s 结束。Host tsdown 的 wall 约为 max(打包, 4.2 s + Typert)。 - **无损依据:** 第二轮的 54 次运行全部以 0 退出,每次的 743 个产物文件均与基线一致(generator 自身的 `lib/index.js` 除外)。测试 `writes the artifacts in-process generation produces` 比较 worker 与进程内生成的产物;在临时工作区中把一个包的 `files` 改坏后,tsdown 在 2.4 s 内以 1 退出,报告 `TypertWorkerError` 及 worker 内的 `TypertAnalysisError`,不写入任何产物。各失败路径由 `typertPlugin workspace mode` 测试组覆盖。 - **取舍:** worker 堆约 300–420 MiB;最终配置的 Host tsdown 最大 RSS 中位数为 6,296–6,408 MiB,基线为 5,676 MiB。共享主机高负载时 worker 可能落在关键路径上:一次样本为 21.91 s,拥有产物的包构建等待 worker 约 14 s,其后 10 个样本未复现;load average 约 100 时两次运行为 17.86 s 与 14.3 s。worker 在 1,024 MiB 堆上限下也能完成,因此不是 GC 压力;归因于主机争用尚未证实。 @@ -97,14 +99,14 @@ Status: implemented - **目的:** 避免每个模块都从 rolldown 进入 JavaScript `transform` 钩子,只为被扩展名检查拒绝。 - **改动:** `transform` 声明 `filter: { id: /^[^?]*\.[cm]?tsx?(?:\?|$)/ }`,处理函数仍以同一正则检查 id。 -- **性能:** 每次 Host 构建的钩子调用从 1,566 次降到 195 次(195 个 TypeScript 模块,其中 4 个含装饰器)。wall 为 11.07 s(11.07 / 10.21 / 11.30)与 9.98 s(4 个样本),第 6 项为 10.25 s,相同代码的对照组为 11.07 s 与 9.87 s,差异在噪声内。 +- **性能:** 每次 Host 构建的钩子调用从 1,566 次降到 195 次(195 个 TypeScript 模块,其中 4 个含装饰器)。wall 中位数为 11.07 s(3 个样本)与 9.98 s(4 个样本),第 6 项为 10.25 s,相同代码的对照组为 11.07 s 与 9.87 s,差异在噪声内。 - **无损依据:** 该正则只含 ASCII 构造,在 rolldown 的 Rust 正则与 JavaScript 中匹配相同的 id 集合;处理函数应用同一检查,因此不读取过滤器的宿主行为相同。产物逐字节一致;测试 `filters transform calls to TypeScript module ids` 固定过滤器。 ### 8. Client tsdown 预设按包目录读取 manifest - **目的:** 每个包配置只读取自己的 `package.json`,消除按包名遍历全部 manifest 的 glob。 - **改动:** [`packages/client/tsdown.client.ts`](../../../../packages/client/tsdown.client.ts) 的 `clientBundle`、`staticLinked` 与 `clientLibrary` 的第一个参数是包目录(通常为配置文件的 `import.meta.dirname`),预设读取一次 `/package.json`,以其中的 `name` 作为插件 id 与诊断名称;该 manifest 缺失、不可读或缺少 `name` 时,配置加载以指明路径与原因的错误失败。82 个包配置调用这些预设。预设不保留模块级 manifest 缓存;Node 半边的生产依赖 external 列表在首次 resolve 时由该 manifest 计算,并缓存在该配置的闭包中。 -- **性能:** Client tsdown 从 12.78 s(13.77 / 12.78 / 12.31)降到 8.73 s(8.19 / 8.80 / 8.73),−4.05 s(−31.7%),两组样本区间不重叠;user CPU 从 26.1 s 降到 25.2 s,sys 从 5.9 s 降到 4.9 s,因此收益主要来自解除主线程阻塞,而不是减少 CPU 总量。原型(一次 glob 建立按包名索引)为 11.72 s → 7.46 s(−36%)。Host tsdown 中只有 5 个 `hostPhase` 包经过该查找,一对样本为 11.92 s → 10.71 s,置信度低。 +- **性能:** Client tsdown 从 12.78 s 降到 8.73 s(各 3 个样本的中位数),−4.05 s(−31.7%),两组样本区间不重叠;user CPU 从 26.1 s 降到 25.2 s,sys 从 5.9 s 降到 4.9 s,因此收益主要来自解除主线程阻塞,而不是减少 CPU 总量。原型(一次 glob 建立按包名索引)为 11.72 s → 7.46 s(−36%)。Host tsdown 中只有 5 个 `hostPhase` 包经过该查找,一对样本为 11.92 s → 10.71 s,置信度低。 - **无损依据:** 11 次运行的 11,566 个 lib 文件逐字节一致。82 个调用处改为传入包目录后,没有一个解析出的包名与原先手写的包名不同。进程内比较 82 个配置 × 3 个 face × 1,600 个 specifier 的 external 判定,结果与原实现完全一致;负对照(多加一个 external)使 75 个判定不同。`client-bundle-purity.spec.ts` 覆盖这些 manifest 错误。 - **行为差异:** 预设的调用约定从包名改为包目录。 @@ -125,7 +127,7 @@ Status: implemented ## 后果 -Client 类型检查的 CPU 与 wall 时间、Host tsdown 与 Client tsdown 的 wall 时间都明显下降,且不改变 runtime 解析、发布安装语义或任何产物字节。一次完整 `build` 中,Host tsdown 为 11.4 s、Client tsdown 为 9.0 s、Client 聚合检查为 34.8 s,master 串行构建中依次为 22.5、13.9 与 68.2 s(各一个样本,见[构建拆分 Note](2026-10-11-build-src-test-split.zh.md))。Host tsdown 位于 `build` 与 `build:src` 的关键路径上,Client tsdown 位于 `build:src` 的关键路径上,Client 聚合检查是 `build` 关键路径的最后一步。 +Client 类型检查的 CPU 与 wall 时间、Host tsdown 与 Client tsdown 的 wall 时间都明显下降,且不改变 runtime 解析、发布安装语义或任何产物字节。串行构建的各段中(master → 本分支,各一个样本),Host tsdown 从 22.5 s 降到 11.7 s,Client 包编译从 55.6 s 降到 48.7 s,Client 聚合检查从 68.2 s 降到 35.9 s,Client tsdown 从 13.9 s 降到 10.5 s;串行各段合计在 master 上为 308.5 s,在本分支上为 254.9 s(见[构建拆分 Note](2026-10-11-build-src-test-split.zh.md))。Host tsdown 位于 `build` 与 `build:src` 的关键路径上,Client tsdown 位于 `build:src` 的关键路径上,Client 聚合检查是 `build` 关键路径的最后一步。 代价:所有 workspace 包共用一个 React 类型版本;Typert 不校验它不分析的包 tsconfig,这些错误由 `tsc -b` 负责;Host tsdown 的最大 RSS 约增加 0.6 GiB,共享主机高负载时 worker 可能成为 Host tsdown 的关键路径。 diff --git a/docs/development.i18n.yaml b/docs/development.i18n.yaml index 0e37edd824..d06ec1fc49 100644 --- a/docs/development.i18n.yaml +++ b/docs/development.i18n.yaml @@ -21,7 +21,7 @@ en: 3221d1076554001f zh: 29f166be6a5c04d9 /development-guide/contributor-reference/typescript-project-layout: - en: 95cea426e6286cb8 + en: 0efec3344c2687b0 zh: 155d5e5564b07952 /development-guide/contributor-reference/environment-variables: en: dba2c72ee6ffa930 diff --git a/docs/development.md b/docs/development.md index e88e39ba59..4d6a34a8a2 100644 --- a/docs/development.md +++ b/docs/development.md @@ -94,7 +94,7 @@ Both tsdown passes match `vendor/*`, `packages/*/*`, and `apps/cli`; the Host pa Typert runs only during Host tsdown, seeded by `tsconfig.host.json`. It analyzes Host types and generates both Host reflection artifacts and the Host-for-Client Remote projection; Client tsdown does not start Typert. Consequently, Client package compilation always follows the Host bundle, and `pnpm run typecheck` runs the complete Host lib phase, including the Host aggregate check, before Client tsc. `lint`, `lint:fix`, and `doc-typecheck` read the generated declarations but not the Host aggregate check, so they prepare only Host package compilation and the Host bundle (`compile-referenced-projects.ts host-libraries`). -`pnpm run build` and `build:src` embed the root package version, the seven-character source commit, and a dirty marker when Git reports local changes; it also inherits other caller-supplied `DSH_CLIENT_*` values. `pnpm run build:official` is the cross-platform local equivalent of the CI and release artifact build and omits the local dirty marker. Each successful `build` or `build:src` writes a gitignored record that binds the exact public values to the Vite output and dynamic client bundles; release packing and built Web tests reject a missing record or artifacts changed by a later partial build. `pnpm run dev:web` runs `build:src` first (`--skip-build` reuses an existing artifact tree instead), then samples the current version and Git state once and shares that environment across every watcher stage for the session; it does not validate the complete-build record because the watcher stages rewrite its recorded artifacts. +`pnpm run build` and `build:src` embed the root package version, the seven-character source commit, and a dirty marker when Git reports local changes; they also inherit other caller-supplied `DSH_CLIENT_*` values. `pnpm run build:official` is the cross-platform local equivalent of the CI and release artifact build and omits the local dirty marker. Each successful `build` or `build:src` writes a gitignored record that binds the exact public values to the Vite output and dynamic client bundles; release packing and built Web tests reject a missing record or artifacts changed by a later partial build. `pnpm run dev:web` runs `build:src` first (`--skip-build` reuses an existing artifact tree instead), then samples the current version and Git state once and shares that environment across every watcher stage for the session; it does not validate the complete-build record because the watcher stages rewrite its recorded artifacts. Static analysis and tests resolve workspace imports through the base `paths` map to `src` and must pass on a clean tree; gates that consume built `lib/` output declare that dependency explicitly. Generated Host-for-Client Remote declarations are the deliberate exception: the public `typecheck`, `lint`, and `doc-typecheck` commands generate them first, while internal `*:contracts-ready` scripts assume that an invoking public command or scheduler gate already depends on the Typert contract-generation pass or the complete build. See the [Typert Remote note](../.agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.md) for the gate-preparation contract. diff --git a/packages/client/AGENTS.md b/packages/client/AGENTS.md index 8640f6dba0..16956ca71d 100644 --- a/packages/client/AGENTS.md +++ b/packages/client/AGENTS.md @@ -69,7 +69,7 @@ Npm sections describe installation and development relationships; each build fac 5. **Only Host runtime libraries require installation dependencies.** Browser-only third-party implementations belong in `devDependencies`, whether inlined into `lib/client.js` or retained as bare imports in a static browser library. Static libraries are Web-shell build inputs; the final Vite build merges their imports and styles. Distributed browser code remains subject to runtime license disclosure ([reference](../../.agents/notes/implemented/process/2026-08-26-published-dependency-faces.md)). 6. **Browser and Node build faces declare externality independently.** A dynamic browser half uses the baseline plus `dsh.client.external`; a statically linked face externalizes every bare specifier; a Node face externalizes its production dependencies ([`tsdown.client.ts`](tsdown.client.ts)). Moving a name between npm sections must not silently change bundle contents. 7. **Keep the published payload closed.** Every relative runtime import and emitted asset must be covered by `files`; the repository publint pass checks the exact publication view. -8. **Only the root manifest declares React type packages.** `@types/react` and `@types/react-dom` belong only in the root `package.json` `devDependencies`, and `verify-package-dependencies` rejects them in every workspace manifest; packages still declare the `react` and `react-dom` they import. Each package-local pnpm link is one more location TypeScript scans when it computes module specifiers for every JSX file ([rationale](../../.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.md#1-only-the-root-manifest-declares-react-type-packages)). +8. **Only the root manifest declares React type packages.** `@types/react` and `@types/react-dom` belong only in the root `package.json` `devDependencies`, and `verify-package-dependencies` rejects them in every manifest under `apps/*`, `packages/*/*`, and `vendor/*`; packages still declare the `react` and `react-dom` they import. Each package-local pnpm link is one more location TypeScript scans when it computes module specifiers for every JSX file ([rationale](../../.agents/notes/implemented/process/2026-10-11-lossless-build-speedups.md#1-only-the-root-manifest-declares-react-type-packages)). ## Build-time browser environment