fix(hmr): preserve native manifest behavior

This commit is contained in:
imccyu committed 2026-10-02 13:53:41 +08:00
1 parent 6fe4a2e184
commit 980b410340
11 files changed
+288 -102

No files matched your search

@@ -6,26 +6,26 @@
en: e6905d7657387304
zh: a7e906173a82f27c
/agent-note-expire-hmr-package-configuration/problem:
en: 0c94c90beb8f27da
zh: 721bec27065b1c52
en: e327f239592ffec0
zh: 339aa414b9dc8652
/agent-note-expire-hmr-package-configuration/decision:
en: a15c7c13f167e5ac
zh: a15c7c13f167e5ac
/agent-note-expire-hmr-package-configuration/decision/responsibilities:
en: 8e2c32e7bd3603fd
zh: 702ce5a1c5a45ccd
en: e72abbf8fafc89bb
zh: f6c59177696c5cd3
/agent-note-expire-hmr-package-configuration/decision/hmr-package-configuration-expiry:
en: d0becbacae5038cc
zh: 8d20bce7ccc84152
/agent-note-expire-hmr-package-configuration/decision/unsupported-scenarios:
en: 0386bb3af5765b44
zh: 3206e2d2286b8c9c
en: e5677d53b4aa8c1b
zh: 3aa660a2d13c63e2
/agent-note-expire-hmr-package-configuration/decision/future-work:
en: a65f33aa459568ed
zh: b75c5eb69420ca2d
/agent-note-expire-hmr-package-configuration/alternatives-considered:
en: de12dc08fc9c52bb
zh: 3385a252091626a3
en: 47590f8e9712ab5f
zh: 681162750e3b5cfb
/agent-note-expire-hmr-package-configuration/verification:
en: 31671100a78f896e
zh: db0c6976c2615257
en: eecb458e1c7e1b3e
zh: ec2a3e1633d5427a
/agent-note-expire-hmr-package-configuration/consequences:
en: 16dd88467b324ab2
zh: 0f59b55846e27ca5
en: e557562e3f7b07bc
zh: 3cd196a63e4099b0
@@ -6,7 +6,7 @@ English | [中文](2026-09-30-profile-package-refresh-and-manifest-invalidation.
## Problem
- **HMR handled module contents but not package.json.** `packages/boot/hmr` clears module caches on a source change and re-imports the loaded URL. A plugin directory's package.json is not a module: HMR ignored its change and kept Node's package-configuration caches (the C++ package.json reader cache, `package_json_reader`'s nearest-manifest cache, the ESM `ResolveCache`, and CommonJS `_pathCache`). Later reloads and imports kept the old exports, main, and imports, and format detection kept the old type.
- **Package configuration and module evaluation have separate caches.** Clearing a plugin's module cache does not refresh the package.json fields that Node uses for exports, main, imports, and format detection. A package.json can also be imported as a JSON module, in which case its consumers need ordinary module reloads.
## Decision
@@ -14,40 +14,44 @@ English | [中文](2026-09-30-profile-package-refresh-and-manifest-invalidation.
| Owner | Does | Does not |
|---|---|---|
| HMR (`packages/boot/hmr`) | Expires a package's configuration caches when a package.json inside its watched roots changes | Reload plugins for a package.json change; handle `node_modules` |
| HMR (`packages/boot/hmr`) | Expires package configuration and preserves ordinary reloads for manifests loaded as JSON modules | Reload plugins for configuration-only manifests; handle `node_modules` package replacement |
### HMR package-configuration expiry
In HMR's change dispatch, a changed file named `package.json` outside `node_modules` goes to `PackageManifests.invalidate()` instead of `partialReload`. Source changes in the same batch reload afterwards and read the new configuration.
In HMR's change dispatch, a changed file named `package.json` outside `node_modules` goes to `PackageManifests.invalidate()`. A manifest in the host dependency graph still requests a host reload; one loaded as a JSON module still reloads its consumers. A configuration-only manifest schedules no module reload. Configuration-owned paths stay with their dedicated watcher. Source changes in the same batch reload afterwards.
`invalidate(manifest)` records the directory as expired. Afterwards:
- package.json reads, scope lookups, type lookups, and nearest-manifest lookups below that directory read the current file. Ownership follows the real directory when a consumer reaches the package through a link.
- ESM `ResolveCache` entries whose parent lies in that directory or whose result points into it are deleted.
- CommonJS `_pathCache` entries whose result or lookup path lies in that directory are deleted.
- ESM `ResolveCache` is cleared. Its entries do not record every consulted manifest, and a package entry may resolve outside the package directory.
- CommonJS `_pathCache` is cleared for the same reason. Unrelated requests recompute their resolution without unloading their modules.
Loaded modules stay unchanged. A reloaded plugin is re-imported from its loaded URL, so an entry rename takes effect when the Loader entry restarts, for example after disabling and enabling it, when the Loader resolves the package name again.
Configuration invalidation alone leaves loaded modules unchanged. Stopping and restarting a Loader entry resolves its package name again and selects the new entry. Tracking active entry URLs independently of later package-name resolutions is deferred.
The implementation lives in `packages/boot/hmr/src/package-manifest.ts`, which encapsulates every Node internal interface it needs; `index.ts` adds only the dispatch branch, a field, and restoration on disposal. HMR's `node_modules` exclusion is unchanged.
The implementation lives in `packages/boot/hmr/src/package-manifest.ts`, which encapsulates its Node internal interfaces. `index.ts` dispatches manifest changes without changing the module-reload algorithm. HMR's `node_modules` exclusion is unchanged.
| Interface | Action |
|---|---|
| modules binding `readPackageJSON`, `getPackageScopeConfig`, `getPackageType` | Replaced: paths in an expired directory read the current file; other calls reach the native method |
| `package_json_reader.getNearestParentPackageJSON` | Replaced: expired directories bypass its JS cache; a lookup without a manifest returns the native absent result |
| The ESM Loader's `ResolveCache` instance | The prototype `get` is replaced for one lookup to obtain the instance and restored at once; entries are then deleted by parent |
| CommonJS `Module._pathCache` | Entries below an expired directory are deleted |
| The ESM Loader's `ResolveCache` instance | The prototype `get` is replaced for one lookup to obtain the instance and restored at once; the resolution cache is then cleared |
| CommonJS `Module._pathCache` | The request-to-filename cache is cleared |
The binding's `getNearestParentPackageJSON` is called only by `package_json_reader` and is not replaced. The replacements are installed at the first expiry and restored when the HMR service is disposed.
### Unsupported scenarios
### Future Work
Overwriting or upgrading a package installed in `node_modules`, and reinstalling at the same path after removal, still require restart. Module caches, `node_modules` package-configuration caches, and HMR's `node_modules` exclusion are unchanged.
- Online replacement inside `node_modules`, same-path reinstall, changed link targets, and cross-package reload propagation remain unsupported. Package updates need process restart; this does not guarantee that every management result already reports that requirement correctly.
- TSX versions using an asynchronous loader thread keep that thread's package configuration outside these hooks. Synchronizing it is deferred; this change does not provide general Worker cache synchronization.
- After exports or main moves an entry, HMR can fail to locate its old loaded module by package name. Keeping an active-entry URL association is deferred; restarting the Loader entry selects the new entry.
- An already loaded CommonJS request can still hit Node's private request cache and return its existing module even after `require.resolve()` changes. Invalidating that request cache and replacing module instances are deferred.
- Disposing HMR restores the native readers, whose previous cached configuration can become visible again. Preserving invalidation state across HMR replacement is deferred.
## Alternatives considered
**Let HMR reload packages inside `node_modules`.** It requires finding affected plugins over a module graph that includes `node_modules`, preventing duplicate evaluation of shared libraries and Cordis, and handling missing dynamic-import edges, module side effects, and Workers. It is not done here; those scenarios keep requiring restart.
**Reload plugins when package.json changes.** package.json is not a module, and changing it does not necessarily call for re-evaluation; source HMR is driven by source changes. After expiry, the next source reload or new import reads the new configuration.
**Reload every plugin for a package.json change.** Configuration-only manifests do not require module evaluation. Manifests actually imported as JSON modules retain ordinary dependency-driven reloads.
**Re-import reloaded plugins by package name.** It would let an entry rename follow a source reload, but changes `partialReload`'s rule that the loaded URL is the reload unit. Entry renames already take effect when the Loader entry restarts, so this is not changed.
@@ -56,12 +60,12 @@ Overwriting or upgrading a package installed in `node_modules`, and reinstalling
| Coverage | Location |
|---|---|
| Expired exports, main, imports, type, scope, and nearest manifests; parity with the native reader; the `node_modules` boundary; restoration | `packages/boot/hmr/tests/package-manifest.spec.ts` |
| HMR dispatch: a manifest change alone reloads nothing, a same-batch source reload reads the new imports, an entry restart reads the new entry, and `node_modules` manifests are ignored | `packages/boot/hmr/tests/package-manifest-dispatch.spec.ts` |
| Configuration-only manifests, JSON-module and host reloads, source reloads, entry restarts, and the `node_modules` exclusion | `packages/boot/hmr/tests/package-manifest-dispatch.spec.ts` |
Tests need no API key and make no model calls.
## Consequences
- During HMR development, changes to package.json imports, exports, main, and type reach the next reload or new import; package.json alone triggers no reload.
- In the supported thread, package.json configuration reads use the changed fields; JSON modules retain their own reload behavior. The separate module, loader-thread, and Worker limitations above still apply.
- Package configuration below expired directories is parsed in JavaScript and must keep the native reader's field and error semantics; tests compare it with the native reader on Node 22, 24, and 26.
- The implementation depends on several Node internal interfaces, and these tests must run again for Node upgrades.
@@ -6,7 +6,7 @@ Status: implemented
## Problem
- **HMR 只处理模块内容,不处理 package.json。** `packages/boot/hmr` 在源码变化时清模块缓存,并按原 URL 重新 import。插件目录里的 package.json 不是模块,改了它,HMR 既不处理这个事件,也不清 Node 的包配置缓存(C++ 的 package.json 读取缓存、`package_json_reader` 的最近 package.json 缓存、ESM `ResolveCache`、CJS `_pathCache`)。之后的源码重载和新导入仍按旧的 exports、main、imports 解析,按旧的 type 判断格式。
- **包配置与模块求值使用独立缓存。** 清理插件的模块缓存不会刷新 Node 用于 exports、main、imports 和格式判断的 package.json 字段。package.json 也可以被作为 JSON 模块导入,此时其消费方需要普通的模块重载。
## Decision
@@ -14,40 +14,44 @@ Status: implemented
| 负责方 | 做什么 | 不做什么 |
|---|---|---|
| HMR(`packages/boot/hmr`) | 监听范围内 package.json 变化时,让该包的配置缓存失效 | 不因 package.json 变化重载插件;不处理 `node_modules` |
| HMR(`packages/boot/hmr`) | 使包配置失效,并保留作为 JSON 模块加载的 manifest 的普通重载 | 不因仅作为配置的 manifest 重载插件;不处理 `node_modules` 包替换 |
### HMR 包配置失效
HMR 的文件变化分派中,文件名为 `package.json` 且不在 `node_modules` 内的变化,交给 `PackageManifests.invalidate()`,不进 `partialReload`。同一批里的源码变化随后照常重载,重载后的模块读到新配置。
HMR 的文件变化分派中,文件名为 `package.json` 且不在 `node_modules` 内的变化,交给 `PackageManifests.invalidate()`。位于宿主依赖图中的 manifest 仍请求宿主重载;作为 JSON 模块加载的 manifest 仍重载消费方。仅作为配置的 manifest 不安排模块重载。配置专属路径仍由其独立 watcher 处理。同一批的源码变化随后重载。
`invalidate(manifest)` 把该目录登记为已失效,之后:
- 位于该目录下的 package.json 读取、scope 查询、type 查询、最近 package.json 查询,都按当前磁盘解析。消费方经由链接路径访问时,按真实目录判断归属。
- ESM `ResolveCache` 中,parent 位于该目录、或结果指向该目录的条目被删除。
- CJS `_pathCache` 中,结果或查找路径位于该目录的条目被删除。
- 清空 ESM `ResolveCache`。其条目不记录查询过的全部 manifest,包入口也可能解析到包目录之外。
- 基于相同原因清空 CommonJS `_pathCache`。无关请求重新计算解析结果,不卸载其模块。
已加载模块保持不变。重载的插件按已加载的 URL 重新 import,因此入口文件改名在 Loader entry 重新启动(例如停用后再启用)时生效,那时 Loader 按包名重新解析。
仅使配置失效不会改变已加载模块。停止并重新启动 Loader entry 时,会重新按包名解析并选择新入口。独立于后续包名解析记录活动入口 URL 留待后续。
实现位于 `packages/boot/hmr/src/package-manifest.ts`,所需的 Node internal 接口都封装在这个文件里;`index.ts` 只增加分派、字段和卸载时的恢复。`node_modules` 的排除规则不变。
实现位于 `packages/boot/hmr/src/package-manifest.ts`,封装其使用的 Node internal 接口。`index.ts` 分派 manifest 变更,不改变模块重载算法。HMR 的 `node_modules` 排除规则不变。
| 接口 | 动作 |
|---|---|
| modules binding 的 `readPackageJSON`、`getPackageScopeConfig`、`getPackageType` | 替换:路径位于已失效目录时按当前磁盘解析,其余调用原生方法 |
| `package_json_reader.getNearestParentPackageJSON` | 替换:已失效目录绕过它的 JS 缓存;没有 manifest 时返回原生的缺失结果 |
| ESM Loader 的 `ResolveCache` 实例 | 临时替换原型的 `get`,取得实例后立即恢复,再按 parent 删除条目 |
| CJS `Module._pathCache` | 删除落在已失效目录下的条目 |
| ESM Loader 的 `ResolveCache` 实例 | 临时替换原型的 `get`,取得实例后立即恢复,再清空解析缓存 |
| CJS `Module._pathCache` | 清空请求到文件名的缓存 |
binding 的 `getNearestParentPackageJSON` 只被 `package_json_reader` 自己调用,不替换。这些替换在第一次失效时安装,HMR 服务卸载时恢复。
### 不支持的场景
### 后续工作
覆盖或升级 `node_modules` 里已安装的包、卸载后在同一路径重装,仍然需要重启。模块缓存、`node_modules` 的包配置缓存和 HMR 的 `node_modules` 排除规则都不变。
- 不支持 `node_modules` 内在线替换、同路径重装、link 换目标及跨包重载传播。包更新需要重启进程;这不保证每个管理操作的结果都已正确报告这一要求。
- 使用异步 loader 线程的 TSX 版本,其线程内包配置不受这些 hook 管理。线程同步留待后续;本次不提供通用 Worker 缓存同步。
- exports 或 main 改变入口后,HMR 可能无法按包名找到旧的已加载模块。保留活动 entry URL 关联留待后续;重启 Loader entry 会选择新入口。
- 已加载的 CommonJS 请求仍可能命中 Node 私有请求缓存,即使 `require.resolve()` 改变也返回原模块。请求缓存失效及模块实例替换留待后续。
- 销毁 HMR 会恢复原生 reader,其此前缓存的配置可能重新可见。跨 HMR 替换保留失效状态留待后续。
## Alternatives considered
**让 HMR 支持 `node_modules` 内的包重载。** 这需要沿包含 `node_modules` 的模块图确定受影响插件,防止共享库和 Cordis 被重复求值,还要处理动态 import 漏边、模块副作用和 Worker。本次不做,这些场景继续要求重启。
**package.json 变化时重载插件。** package.json 不是模块,改它不一定意味着需要重新求值;源码 HMR 本来就由源码变化驱动。失效之后,下一次源码重载或新导入自然会读到新配置。
**package.json 变化时重载所有插件。** 仅作为配置的 manifest 不要求模块求值。实际作为 JSON 模块导入的 manifest 保留普通的依赖驱动重载。
**按插件名重新 import 被重载的插件。** 这能让入口改名随源码重载生效,但会改变 `partialReload` 以已加载 URL 为重载单位的规则。入口改名在 Loader entry 重启时已经生效,所以本次不改。
@@ -56,12 +60,12 @@ binding 的 `getNearestParentPackageJSON` 只被 `package_json_reader` 自己调
| 覆盖面 | 位置 |
|---|---|
| 失效后的 exports、main、imports、type、scope、最近 package.json,与原生读取的一致性,node_modules 边界,恢复 | `packages/boot/hmr/tests/package-manifest.spec.ts` |
| HMR 分派:仅 manifest 变化不重载、同批源码重载读新 imports、entry 重启读新入口、node_modules 内不处理 | `packages/boot/hmr/tests/package-manifest-dispatch.spec.ts` |
| 配置专用 manifest、JSON 模块与宿主重载、源码重载、entry 重启和 node_modules 排除规则 | `packages/boot/hmr/tests/package-manifest-dispatch.spec.ts` |
测试不需要 API key,不调用模型。
## Consequences
- HMR 开发流程中改 package.json 的 imports、exports、main、type,下一次重载或新导入就会读到;package.json 本身不触发重载。
- 在支持的线程内,package.json 配置读取使用更新后的字段;JSON 模块保留自己的重载行为。上述模块、loader 线程及 Worker 的独立限制仍然适用。
- 包配置读取改由 JS 解析 package.json,需要与原生读取保持字段和错误语义一致;测试在 Node 22、24、26 上与原生读取逐项比较。
- 依赖多个 Node internal 接口,Node 升级时需要重跑这些测试。
+4 -4
View File
@@ -21,8 +21,8 @@
en: 45ee194ad781d603
zh: b3d7fcd8ed73717c
/deepseek-ai-dsh-hmr/understand-the-implementation:
en: 9b2724d4cedcd1a0
zh: 863bf05e3b115c05
en: 22cc23a232d1a4c0
zh: 1775437af183d999
/deepseek-ai-dsh-hmr/model-experience:
en: 215e7ba838619b7b
zh: a311da3843709f90
@@ -39,8 +39,8 @@
en: 89e0624fafd701fb
zh: 19d36a33c1d9af9d
/deepseek-ai-dsh-hmr/known-limitations-and-deferred-work:
en: dcb1f29ee2e973d4
zh: 520bc58f2e7de42e
en: ebfd5ba2a83aea31
zh: 802474ad532b9002
/deepseek-ai-dsh-hmr/known-limitations-and-deferred-work/dev-note:
en: 7972e4b2005ec9a4
zh: 754c49ec538c164c
+9 -1
View File
@@ -56,7 +56,11 @@ Chokidar options, including polling, retain their existing meaning. Exact config
`watchConfig()` registers an awaited configuration handler. `runExclusive()` serializes configuration changes and Loader updates with automatic reloads and rejects nested transactions. Package installation and removal run outside this queue. HMR does not acquire the package writer lock; manifest notifications reload only when the ordered `dsh.profile.bundles` list changes. Profile and home patch changes also trigger recomposition. File events received during a configuration transaction are processed afterward. Include refreshes and profile reconciliation reach plugins through ordinary Loader entry updates; Loader commits volatile-only changes in place.
App-boot owns profile parsing and patch precedence. HMR reads the launcher’s data-only `profileContext`, registers the profile manifest and both user patch watches during initialization, and waits for application readiness before processing changes. Its disposal closes the watchers and cancels reloads waiting for startup. HMR also owns module-cache replacement and reload scheduling. A changed `package.json` outside `node_modules` is not reloaded as a module: HMR expires Node's cached package configuration for that package directory, including package reads, scope, type, and nearest-manifest lookups, ESM `ResolveCache`, and CommonJS `_pathCache` entries, and reloads nothing for it. Later resolutions, including the imports of a plugin reloaded for a source change in the same or a later batch, read the current manifest; a reloaded plugin is re-imported from its loaded URL, so a new entry takes effect when its Loader entry restarts. [package-manifest.ts](src/package-manifest.ts) owns the Node internal interfaces this uses. Configuration watchers start outside the active transaction context so later notifications can enter the queue.
App-boot owns profile parsing and patch precedence. HMR reads the launcher’s data-only `profileContext`, registers the profile manifest and both user patch watches during initialization, and waits for application readiness before processing changes. Its disposal closes the watchers and cancels reloads waiting for startup. Configuration watchers start outside the active transaction context so later notifications can enter the queue.
A changed `package.json` outside `node_modules` expires that package's cached configuration. A configuration-only manifest triggers no module reload; a manifest imported as a JSON module retains dependency-driven reloads, and one in the host dependency graph retains host reloads. Dedicated configuration watches keep their existing ownership. A new package entry takes effect when its Loader entry restarts.
[package-manifest.ts](src/package-manifest.ts) owns the Node internal interfaces for package reads, scope, type, and nearest-manifest lookups. Manifest invalidation clears the ESM `ResolveCache` and CommonJS `_pathCache`: neither records every consulted manifest, and entries may point outside their package directory. Those are resolution caches, not module-evaluation caches; ordinary module replacement remains HMR's separate operation.
Watched module paths use Node ESM resolution's `realpathSync()` spelling, including Windows short directory names, so file events match the module cache.
@@ -87,6 +91,10 @@ Reloading a contributing plugin can change later request prefixes; HMR does not
- Module replacement requires Node loader internals. Framework dependency changes call the host-provided `loader.exit()` hook; HMR itself does not restart the process.
- Replacing installed package versions still requires a restart through Plugin Manager; manifests below `node_modules` keep Node's cached configuration. The browser Client module graph retains its separate browser-side loading mechanism.
- Future work: support TSX's asynchronous loader thread. Main-thread hooks do not invalidate that thread's package configuration; ordinary application-created Workers are outside the supported scope.
- Future work: retain active entry URLs after `exports` or `main` changes. HMR still resolves entry names when locating loaded modules; after an entry moves, source changes may not reload the old entry until the Loader entry restarts.
- Future work: an already loaded CommonJS request can retain its module through Node's private request cache even after `require.resolve()` changes. Package configuration invalidation does not replace that module instance.
- Future work: retain package invalidation across HMR disposal and replacement. Restoring the native readers can expose their older cached configuration again.
- `watchConfig()` resolves when Chokidar reports readiness. On darwin, libuv starts the FSEvents stream afterwards on its own thread, so a write that lands within milliseconds of registration is not reported until the next event in that directory; edits made after startup are unaffected.
### Dev Note
+9 -1
View File
@@ -56,7 +56,11 @@ Chokidar 选项(包括轮询)保持原有含义。精确配置监听同时
`watchConfig()` 注册会被等待的配置处理器。`runExclusive()` 将配置变更、Loader 更新与自动重载串行化,并拒绝嵌套事务。包安装和删除在该队列之外执行。HMR 不获取包操作写锁;manifest 通知仅在有序的 `dsh.profile.bundles` 列表变化时触发重载。profile 与 home patch 变化也会触发重新组合。配置事务期间收到的文件事件在事务结束后处理。 Include 刷新和 profile 重载都通过普通的 Loader 条目更新到达插件;仅 volatile 变化由 Loader 就地提交。
App-boot 负责 profile 解析和 patch 优先级规则。HMR 读取启动器提供的纯数据 `profileContext`,在初始化时注册 profile manifest 和两份用户 patch 的监听,并等待应用就绪后处理更改。销毁 HMR 时会关闭监听器并取消等待启动的重载。HMR 也负责模块缓存替换和重载调度。`node_modules` 之外的 `package.json` 变化不作为模块重载:HMR 让该包目录在 Node 中缓存的包配置失效,包括 package 读取、scope、type 与最近 manifest 查询、ESM `ResolveCache` 和 CommonJS `_pathCache` 条目,但不因此重载任何插件。之后的解析,包括同一批或之后因源码变化重载的插件的 import,都读取当前 manifest;重载的插件按已加载的 URL 重新 import,因此新入口在其 Loader 条目重启时生效。[package-manifest.ts](src/package-manifest.ts) 封装这里用到的 Node internal 接口。配置监听器在当前事务上下文之外启动,使后续通知可以进入队列。
App-boot 负责 profile 解析和 patch 优先级规则。HMR 读取启动器提供的纯数据 `profileContext`,在初始化时注册 profile manifest 和两份用户 patch 的监听,并等待应用就绪后处理更改。销毁 HMR 时会关闭监听器并取消等待启动的重载。配置监听器在当前事务上下文之外启动,使后续通知可以进入队列。
`node_modules` 之外的 `package.json` 变化使该包的配置缓存失效。仅作为配置的 manifest 不触发模块重载;作为 JSON 模块导入的 manifest 保留依赖驱动重载,位于宿主依赖图中的 manifest 保留宿主重载。专属配置监听保留原有归属。新包入口在 Loader entry 重启时生效。
[package-manifest.ts](src/package-manifest.ts) 封装 package 读取、scope、type 和最近 manifest 查询所需的 Node internal 接口。manifest 失效会清空 ESM `ResolveCache` 和 CommonJS `_pathCache`:它们都不记录查询过的全部 manifest,入口也可能指向包目录之外。这些是解析缓存,不是模块求值缓存;普通模块替换仍由 HMR 独立处理。
被监听模块的路径沿用 Node ESM 解析所用的 `realpathSync()` 表示,包括 Windows 短目录名,使文件事件与模块缓存匹配。
@@ -87,6 +91,10 @@ App-boot 负责 profile 解析和 patch 优先级规则。HMR 读取启动器提
- 模块替换需要 Node loader 内部接口。框架依赖变化调用宿主提供的 `loader.exit()` 钩子;HMR 本身不重启进程。
- 通过插件管理器替换已安装包版本仍需要重启;`node_modules` 之下的 manifest 保留 Node 缓存的配置。浏览器 Client 模块图保留独立的浏览器侧加载机制。
- 后续工作:支持 TSX 的异步 loader 线程。主线程 hook 不会使该线程内的包配置失效;普通应用自行创建的 Worker 不在支持范围内。
- 后续工作:在 `exports` 或 `main` 改变后保留活动 entry URL。HMR 定位已加载模块时仍重新解析 entry 名称;入口移动后,源码变化可能无法重载旧入口,需要先重启 Loader entry。
- 后续工作:已加载的 CommonJS 请求可能通过 Node 私有请求缓存保留原模块,即使 `require.resolve()` 已改变。包配置失效不替换该模块实例。
- 后续工作:跨 HMR 销毁和替换保留包配置失效状态。恢复原生 reader 后,其旧配置缓存可能重新可见。
- `watchConfig()` 在 Chokidar 报告就绪时 resolve。darwin 上 libuv 随后才在自己的线程启动 FSEvents 流,因此注册后数毫秒内落地的写入要等到该目录的下一个事件才会被报告;启动之后的编辑不受影响。
### 开发备注
+4 -4
View File
@@ -279,20 +279,20 @@ class Hmr extends Service {
const filename = canonicalPath(resolve(watchBaseDir, path))
const configuredFilename = resolve(this.baseDir, path)
if (this.configPaths.has(filename) || this.configPaths.has(configuredFilename)) continue
// Package configuration is not a module: later resolutions read it again, but nothing reloads for it.
if (basename(filename) === 'package.json' && !filename.includes(`${sep}node_modules${sep}`)) {
const isManifest = basename(filename) === 'package.json' && !filename.includes(`${sep}node_modules${sep}`)
if (isManifest) {
this.manifests.invalidate(filename)
continue
}
const url = pathToFileURL(filename).href
if (this.externals.has(url)) {
fullReload = true
continue
}
if (this.internal.loadCache.has(url)) {
if (this.internal.loadCache.has(url) || this.internal.loadCache.has(url, 'json')) {
this.stashed.add(url)
continue
}
if (isManifest) continue
const include = [...loader.entries()].map(entry => entry.subtree as Include | undefined)
.find(tree => tree?.filename === filename || tree?.filename === configuredFilename)
if (include !== undefined) includes.add(include)
+22 -21
View File
@@ -123,6 +123,11 @@ interface Configuration {
serialized: PackageTuple
}
interface ImportContext {
base: string
specifier: string | undefined
}
/** Package directories whose manifests are read from disk instead of Node's native cache. */
export class PackageManifests {
private readonly directories = new Set<string>()
@@ -144,33 +149,24 @@ export class PackageManifests {
const directory = dirname(manifest)
this.directories.add(directory)
this.configurations.clear()
const prefix = directory + sep
for (const key of Object.keys(native.cjs._pathCache)) {
const target = native.cjs._pathCache[key]
if (target !== undefined && (target.startsWith(prefix) || key.includes(prefix))) Reflect.deleteProperty(native.cjs._pathCache, key)
}
this.realDirectories.clear()
// A package's entry may resolve outside its directory; cache keys do not record the manifests they consulted.
for (const key of Object.keys(native.cjs._pathCache)) Reflect.deleteProperty(native.cjs._pathCache, key)
const directoryURL = pathToFileURL(directory).href + '/'
const cache = captureEsmResolveCache(native, directoryURL)
/* v8 ignore else -- off-thread loader hooks resolve without this thread's cache; supported launches install none */
if (cache !== undefined) {
// The entry point resolves without a parent URL.
for (const parent of [...Map.prototype.keys.call(cache)] as Array<string | undefined>) {
const entries = Map.prototype.get.call(cache, parent) as Record<string, { url?: string } | undefined>
const stale = parent?.startsWith(directoryURL) === true
|| Object.values(entries).some(result => result?.url?.startsWith(directoryURL) === true)
if (stale) Map.prototype.delete.call(cache, parent)
}
}
/* v8 ignore else -- asynchronous loader hooks own a separate cache; loader-thread invalidation is deferred */
if (cache !== undefined) Map.prototype.clear.call(cache)
}
/** Restore every replaced Node method this instance installed. */
dispose(): void {
for (const restore of this.restorers.splice(0).reverse()) restore()
this.native = undefined
this.realDirectories.clear()
}
private isPathWithinDirectory(path: string, directory: string): boolean {
return path === directory || path.startsWith(directory + sep)
return path === directory || path.startsWith(join(directory, sep))
}
private isPathInInvalidatedDirectory(path: string): boolean {
@@ -216,7 +212,7 @@ export class PackageManifests {
return this.isPathInInvalidatedDirectory(path)
? this.readCachedPackageConfig(
path,
isEsm && base !== undefined ? ` while importing "${specifier}" from ${fileURLToPath(base)}` : '',
isEsm && base !== undefined ? { base, specifier } : undefined,
)?.serialized
: this.rawBinding.readPackageJSON(path, isEsm, base, specifier)
}
@@ -245,8 +241,8 @@ export class PackageManifests {
return { data: found.configuration.data, exists: true, path: found.path }
}
private readCachedPackageConfig(path: string, importing = ''): Configuration | undefined {
if (!this.configurations.has(path)) this.configurations.set(path, this.readPackageConfigFromDisk(path, importing))
private readCachedPackageConfig(path: string, context?: ImportContext): Configuration | undefined {
if (!this.configurations.has(path)) this.configurations.set(path, this.readPackageConfigFromDisk(path, context))
return this.configurations.get(path)
}
@@ -260,12 +256,14 @@ export class PackageManifests {
if (!this.isPathInInvalidatedDirectory(manifest)) return manifest
const configuration = this.readCachedPackageConfig(manifest)
if (configuration !== undefined) return { path: toNamespacedPath(manifest), configuration }
/* v8 ignore next -- root-scope queries run in an isolated child to contain non-termination */
if (parent === directory) return { path: manifest }
directory = parent
}
}
/** @param importing - the native reader's ` while importing "<specifier>" from <path>` suffix, or empty. */
private readPackageConfigFromDisk(path: string, importing: string): Configuration | undefined {
/** @param context - optional ESM import details used only when reporting invalid package configuration. */
private readPackageConfigFromDisk(path: string, context?: ImportContext): Configuration | undefined {
let bytes: Buffer
try { bytes = readFileSync(path) } catch (_error) {
// Node reports a missing manifest as absent configuration.
@@ -273,6 +271,9 @@ export class PackageManifests {
}
const throwInvalidPackageConfig = (): never => {
// The native reader throws a plain Error carrying only this code.
const base = context?.base
const importing = base === undefined ? ''
: ` while importing "${context?.specifier}" from ${base.startsWith('file:') ? fileURLToPath(base) : base}`
throw Object.assign(new Error(`Invalid package config ${toNamespacedPath(path)}${importing}.`), { code: 'ERR_INVALID_PACKAGE_CONFIG' })
}
if (!isUtf8(bytes)) return throwInvalidPackageConfig()
+22
View File
@@ -0,0 +1,22 @@
/** Isolate root-scope traversal; the filesystem root is read but never modified. */
import assert from 'node:assert/strict'
import { createRequire } from 'node:module'
import { join, parse } from 'node:path'
import { pathToFileURL } from 'node:url'
import { PackageManifests } from '../../src/package-manifest.ts'
const [root] = process.argv.slice(2)
const reader = createRequire(import.meta.url)('internal/modules/package_json_reader')
const source = join(root, 'missing', 'entry.js')
const url = pathToFileURL(source).href
const expectedScope = reader.getPackageScopeConfig(url)
const expectedNearest = reader.getNearestParentPackageJSON(source)
const manifests = new PackageManifests()
try {
await manifests.invalidate(join(parse(root).root, 'package.json'))
assert.deepEqual(reader.getPackageScopeConfig(url), expectedScope)
assert.deepEqual(reader.getNearestParentPackageJSON(source), expectedNearest)
process.stdout.write('scope traversal completed\n')
} finally {
manifests.dispose()
}
@@ -40,11 +40,15 @@ function manifest(entry: string, dep: string): string {
return JSON.stringify({ name: 'addon', type: 'module', exports: `./${entry}.mjs`, imports: { '#dep': `./${dep}.mjs` } })
}
async function fixture() {
async function fixture(source = plugin('a')) {
const root = realpathSync.native(mkdtempSync(join(tmpdir(), 'dsh-hmr-manifest-dispatch-')))
const ctx = new Context()
onTestFinished(async () => {
try { await ctx.fiber.dispose() } finally { rmSync(root, { recursive: true, force: true }) }
})
const pkg = join(root, 'plugins', 'addon')
file(join(pkg, 'package.json'), manifest('a', 'dep-1'))
file(join(pkg, 'a.mjs'), plugin('a'))
file(join(pkg, 'a.mjs'), source)
file(join(pkg, 'b.mjs'), plugin('b'))
file(join(pkg, 'dep-1.mjs'), 'export const value = 1')
file(join(pkg, 'dep-2.mjs'), 'export const value = 2')
@@ -52,26 +56,26 @@ async function fixture() {
mkdirSync(join(root, 'app', 'node_modules'), { recursive: true })
symlinkSync(pkg, join(root, 'app', 'node_modules', 'addon'), process.platform === 'win32' ? 'junction' : 'dir')
const trace: string[] = []
const ctx = new Context()
ctx.baseUrl = pathToFileURL(join(root, 'app')).href + '/'
ctx.provide('manifestTrace', trace)
await ctx.plugin(Loader)
await ctx.plugin(Timer)
const start = watchers.length
await ctx.plugin(Hmr, { root: ['../plugins'], ignored: [], debounce: 0 })
onTestFinished(async () => {
await ctx.fiber.dispose()
rmSync(root, { recursive: true, force: true })
})
const watcher = watchers.slice(start).at(-1)!
const entry = ctx.loader.resolve(await ctx.loader.create({ name: 'addon' }))
await ctx.loader.await()
const settle = () => ctx.hmr.runExclusive(async () => {})
const emit = async (...paths: string[]) => {
for (const path of paths) watcher.emit('change', path)
await vi.waitFor(async () => { await settle() })
await new Promise(resolve => setTimeout(resolve, 20))
await settle()
const dispatch = vi.spyOn(ctx.hmr, 'runExclusive')
try {
for (const path of paths) watcher.emit('change', path)
await vi.waitFor(() => { expect(dispatch).toHaveBeenCalledOnce() })
const result = dispatch.mock.results[0]!
expect(result.type).toBe('return')
await result.value
} finally {
dispatch.mockRestore()
}
}
return { ctx, pkg, trace, emit, entry }
}
@@ -89,6 +93,47 @@ it('does not reload for a manifest alone and resolves the new entry when the Loa
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:b:2'])
})
const jsonPlugin = `import manifest from './package.json' with { type: 'json' };
export function apply(ctx) {
ctx.get('manifestTrace').push('start:json:' + manifest.imports['#dep']);
ctx.effect(() => () => { ctx.get('manifestTrace').push('stop:json'); });
}
`
it('reloads a plugin that imports its changed manifest as a JSON module', async () => {
const f = await fixture(jsonPlugin)
expect(f.trace).toEqual(['start:json:./dep-1.mjs'])
file(join(f.pkg, 'package.json'), manifest('a', 'dep-2'))
await f.emit(join(f.pkg, 'package.json'))
expect(f.trace).toEqual(['start:json:./dep-1.mjs', 'stop:json', 'start:json:./dep-2.mjs'])
})
it('requests a host reload for a changed manifest in the host module graph', async () => {
const f = await fixture(jsonPlugin)
const filename = join(f.pkg, 'package.json')
const externals = Reflect.get(f.ctx.hmr, 'externals') as Set<string>
externals.add(pathToFileURL(filename).href)
const exit = vi.spyOn(f.ctx.loader, 'exit').mockImplementation(() => {})
onTestFinished(() => { exit.mockRestore() })
file(filename, manifest('a', 'dep-2'))
await f.emit(filename)
expect(exit).toHaveBeenCalledOnce()
expect(f.trace).toEqual(['start:json:./dep-1.mjs'])
})
it('leaves a configuration-owned manifest to its dedicated watcher', async () => {
const f = await fixture(jsonPlugin)
const filename = join(f.pkg, 'package.json')
const dispose = await f.ctx.hmr.watchConfig(filename, async () => {})
try {
file(filename, manifest('a', 'dep-2'))
await f.emit(filename)
expect(f.trace).toEqual(['start:json:./dep-1.mjs'])
} finally {
await dispose()
}
})
it('resolves package imports from the changed manifest when a source change reloads the plugin', async () => {
const f = await fixture()
file(join(f.pkg, 'package.json'), manifest('a', 'dep-2'))
+110 -16
View File
@@ -1,9 +1,10 @@
/** Package configuration read by Node's resolver follows an invalidated manifest on disk. */
import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
import { spawn } from 'node:child_process'
import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, unlinkSync, writeFileSync } from 'node:fs'
import { createRequire } from 'node:module'
import { tmpdir } from 'node:os'
import { dirname, join, toNamespacedPath } from 'node:path'
import { pathToFileURL } from 'node:url'
import { fileURLToPath, pathToFileURL } from 'node:url'
import { ModuleLoader } from '@deepseek-ai/cordis-plugin-loader'
import { afterEach, describe, expect, it } from 'vitest'
import { PackageManifests } from '../src/package-manifest.ts'
@@ -11,7 +12,7 @@ import { PackageManifests } from '../src/package-manifest.ts'
const require = createRequire(import.meta.url)
const addon = require('node-addon-require-builtin') as { requireBuiltin(id: string): unknown }
const reader = addon.requireBuiltin('internal/modules/package_json_reader') as {
read: (path: string) => { exists: boolean; type: string }
read: (path: string, options?: { isESM: boolean; base: string; specifier: string }) => { exists: boolean; type: string }
getPackageScopeConfig: (url: string) => { exists: boolean; type: string; pjsonPath: string }
getNearestParentPackageJSON: (path: string) => { data: { type: string }; path: string } | undefined
}
@@ -22,8 +23,8 @@ const binding = (addon.requireBuiltin('internal/bootstrap/realm') as {
internalBinding(id: 'modules'): Record<string, unknown>
}).internalBinding('modules')
const cleanup: Array<() => void> = []
afterEach(() => { for (const dispose of cleanup.splice(0).reverse()) dispose() })
const cleanup: Array<() => void | Promise<void>> = []
afterEach(async () => { for (const dispose of cleanup.splice(0).reverse()) await dispose() })
function file(path: string, source: string): void {
mkdirSync(dirname(path), { recursive: true })
@@ -48,7 +49,8 @@ function fixture() {
}
const manifests = new PackageManifests()
cleanup.push(() => { manifests.dispose() })
return { root, dir, manifest, importer, importerURL: pathToFileURL(importer).href, manifests }
const invalidate = (path: string) => Promise.resolve().then(manifests.invalidate.bind(manifests, path))
return { root, dir, manifest, importer, importerURL: pathToFileURL(importer).href, manifests, invalidate }
}
function resolveEsm(specifier: string, parentURL: string): string {
@@ -59,6 +61,26 @@ function resolveEsm(specifier: string, parentURL: string): string {
}
describe('package manifest invalidation', { concurrent: false }, () => {
it.each(['symlink', 'main'] as const)('refreshes a cached external %s entry after its manifest changes', async (entry) => {
const f = fixture()
const external = join(f.root, 'packages', 'outside', 'entry.mjs')
file(external, 'export {}')
if (entry === 'symlink') {
symlinkSync(dirname(external), join(f.dir, 'linked'), process.platform === 'win32' ? 'junction' : 'dir')
file(f.manifest, JSON.stringify({ name: 'pkg', exports: './linked/entry.mjs' }))
} else {
file(f.manifest, JSON.stringify({ name: 'pkg', main: external }))
}
const importerRequire = createRequire(f.importer)
expect(importerRequire.resolve('pkg')).toBe(external)
expect(resolveEsm('pkg', f.importerURL)).toBe(pathToFileURL(external).href)
file(f.manifest, JSON.stringify({ name: 'pkg', main: './b.mjs', exports: './b.mjs' }))
await f.invalidate(f.manifest)
expect.soft(importerRequire.resolve('pkg')).toBe(join(f.dir, 'b.mjs'))
expect(resolveEsm('pkg', f.importerURL)).toBe(pathToFileURL(join(f.dir, 'b.mjs')).href)
})
it('resolves changed exports and main only after the manifest is invalidated', () => {
const f = fixture()
file(f.manifest, JSON.stringify({ name: 'pkg', exports: { import: './a.mjs', require: './a.cjs' } }))
@@ -131,6 +153,16 @@ describe('package manifest invalidation', { concurrent: false }, () => {
f.manifests.dispose()
expect(binding.readPackageJSON).toBe(original)
})
it('reports invalid JSON with its non-file ESM base after invalidation', async () => {
const f = fixture()
file(f.manifest, '{')
await f.invalidate(f.manifest)
const base = 'data:text/javascript,export{}'
const failure = outcome(() => reader.read(f.manifest, { isESM: true, base, specifier: 'pkg' }))
expect(failure).toHaveProperty('code', 'ERR_INVALID_PACKAGE_CONFIG')
expect(failure).toHaveProperty('message', expect.stringContaining(base))
})
})
function outcome(run: () => unknown): unknown {
@@ -141,6 +173,58 @@ function outcome(run: () => unknown): unknown {
}
describe('package manifest reader parity', { concurrent: false }, () => {
it('terminates scope lookup after invalidating the filesystem root without modifying it', async () => {
const f = fixture()
const child = spawn(process.execPath, [
'--expose-internals',
fileURLToPath(new URL('./fixtures/package-root-scope.mjs', import.meta.url)),
f.root,
], { stdio: ['ignore', 'pipe', 'pipe'] })
const completed = new Promise<{ code: number | null; signal: NodeJS.Signals | null }>((resolve, reject) => {
child.once('error', reject)
child.once('close', (code, signal) => { resolve({ code, signal }) })
})
cleanup.push(async () => {
if (child.exitCode === null && child.signalCode === null) child.kill()
await completed
})
let stdout = '', stderr = ''
child.stdout.setEncoding('utf8').on('data', (chunk: string) => { stdout += chunk })
child.stderr.setEncoding('utf8').on('data', (chunk: string) => { stderr += chunk })
const result = await completed
expect(result.signal, stderr).toBeNull()
expect(result.code, stderr).toBe(0)
expect(stdout).toBe('scope traversal completed\n')
})
it('reads valid JSON with a non-file ESM base like Node', async () => {
const f = fixture()
file(f.manifest, '{}')
const options = { isESM: true, base: 'data:text/javascript,export{}', specifier: 'pkg' }
const expected = outcome(() => reader.read(f.manifest, options))
await f.invalidate(f.manifest)
expect(outcome(() => reader.read(f.manifest, options))).toEqual(expected)
})
it('refreshes a cached real directory after a package link moves', async () => {
const f = fixture()
const other = join(f.root, 'packages', 'other')
const link = join(f.root, 'importer', 'node_modules', 'pkg')
const linkedManifest = join(link, 'package.json')
file(f.manifest, '{"type":"commonjs"}')
file(join(other, 'package.json'), '{"type":"commonjs"}')
unlinkSync(link)
symlinkSync(other, link, process.platform === 'win32' ? 'junction' : 'dir')
await f.invalidate(f.manifest)
expect(reader.read(linkedManifest).type).toBe('commonjs')
unlinkSync(link)
symlinkSync(f.dir, link, process.platform === 'win32' ? 'junction' : 'dir')
file(f.manifest, '{"type":"module"}')
await f.invalidate(f.manifest)
expect(reader.read(linkedManifest).type).toBe('module')
})
it.each([
'{}',
'{"name":"pkg","main":"./a.cjs","type":"commonjs"}',
@@ -266,19 +350,29 @@ describe('package manifest reader parity', { concurrent: false }, () => {
expect(importerRequire.resolve('pkg')).toBe(join(f.dir, 'b.cjs'))
})
it('keeps readers replaced after installation when disposed', () => {
it('keeps readers replaced after installation when disposed', async () => {
const f = fixture()
file(f.manifest, '{}')
f.manifests.invalidate(f.manifest)
const later = () => undefined
const previousRead = binding.readPackageJSON
const previousScope = binding.getPackageScopeConfig
const previousType = binding.getPackageType
const previousNearest = reader.getNearestParentPackageJSON
binding.readPackageJSON = later
reader.getNearestParentPackageJSON = later
f.manifests.dispose()
expect(binding.readPackageJSON).toBe(later)
expect(reader.getNearestParentPackageJSON).toBe(later)
binding.readPackageJSON = previousRead
reader.getNearestParentPackageJSON = previousNearest
try {
await f.invalidate(f.manifest)
const later = () => undefined
binding.readPackageJSON = later
reader.getNearestParentPackageJSON = later
f.manifests.dispose()
expect(binding.readPackageJSON).toBe(later)
expect(reader.getNearestParentPackageJSON).toBe(later)
expect(binding.getPackageScopeConfig).toBe(previousScope)
expect(binding.getPackageType).toBe(previousType)
} finally {
f.manifests.dispose()
binding.readPackageJSON = previousRead
binding.getPackageScopeConfig = previousScope
binding.getPackageType = previousType
reader.getNearestParentPackageJSON = previousNearest
}
})
})