Merge pull request #5583 from deepseek-harness/fix/hmr-package-manifest-invalidation-1001

fix(hmr): invalidate changed package manifests
This commit is contained in:
imccyu authored and GitHub committed 2026-10-02 17:08:57 +08:00
commit fdacc98bf1
32 files changed
+2180 -65

No files matched your search

@@ -0,0 +1,31 @@
# Bilingual-pair consistency record for 2026-09-30-profile-package-refresh-and-manifest-invalidation.md (docs/i18n/README.md): per heading
# section, a hash of its English and Chinese blocks outside code blocks and generated regions.
# After editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-30-profile-package-refresh-and-manifest-invalidation.md
/agent-note-expire-hmr-package-configuration:
en: e6905d7657387304
zh: a7e906173a82f27c
/agent-note-expire-hmr-package-configuration/problem:
en: e327f239592ffec0
zh: 339aa414b9dc8652
/agent-note-expire-hmr-package-configuration/decision:
en: a15c7c13f167e5ac
zh: a15c7c13f167e5ac
/agent-note-expire-hmr-package-configuration/decision/responsibilities:
en: e72abbf8fafc89bb
zh: f6c59177696c5cd3
/agent-note-expire-hmr-package-configuration/decision/hmr-package-configuration-expiry:
en: c7de8dafe47a5587
zh: 6010a70d8451edd1
/agent-note-expire-hmr-package-configuration/decision/future-work:
en: 31818f8f7f80470b
zh: ecb7ac85bb49cf6d
/agent-note-expire-hmr-package-configuration/alternatives-considered:
en: 5dab1237a312169c
zh: b24927d5badedcf7
/agent-note-expire-hmr-package-configuration/verification:
en: d1d931904beb1c9f
zh: ea7874b974e1c209
/agent-note-expire-hmr-package-configuration/consequences:
en: e557562e3f7b07bc
zh: 3cd196a63e4099b0
@@ -0,0 +1,78 @@
# Agent Note: Expire HMR package configuration
Status: implemented
English | [中文](2026-09-30-profile-package-refresh-and-manifest-invalidation.zh.md)
## Problem
- **Package configuration and module evaluation have separate caches.** Clearing a plugin's module cache does not refresh the package.json fields that Node uses for exports, main, imports, and format detection. A package.json can also be imported as a JSON module, in which case its consumers need ordinary module reloads.
## Decision
### Responsibilities
| Owner | Does | Does not |
|---|---|---|
| HMR (`packages/boot/hmr`) | Expires package configuration and preserves ordinary reloads for manifests loaded as JSON modules | Reload plugins for configuration-only manifests; handle `node_modules` package replacement |
### HMR package-configuration expiry
In HMR's change dispatch, a changed file named `package.json` outside `node_modules` goes to `PackageManifests.invalidate()`. A manifest in the host dependency graph still requests a host reload; one loaded as a JSON module still reloads its consumers. A configuration-only manifest schedules no module reload. Configuration-owned paths stay with their dedicated watcher. Source changes in the same batch reload afterwards.
`invalidate(manifest)` records the directory as expired. Afterwards:
- package.json reads, scope lookups, type lookups, and nearest-manifest lookups below that directory read the current file. Ownership follows the real directory when a consumer reaches the package through a link.
- ESM `ResolveCache` is cleared. Its entries do not record every consulted manifest, and a package entry may resolve outside the package directory.
- CommonJS `_pathCache` is cleared for the same reason. Unrelated requests recompute their resolution without unloading their modules.
- CommonJS loads using the default resolver pass the freshly resolved filename to the native loader. Its private request alias cannot select an older entry; cached module instances remain intact.
Configuration invalidation alone leaves loaded modules unchanged. Loader entries retain their raw import results before export normalization. HMR matches imported Node module objects to cached ModuleJob namespaces and reloads the loaded URLs. All matching entry records, including disabled entries, update only after a successful reload and remain unchanged on failure. Stopping and restarting a Loader entry resolves its package name again and selects the new entry.
Entry names are scoped by configuration-tree base URL and retain all distinct imported namespaces. An uninitialized entry contributes the name without erasing loaded namespaces; name-based resolution is used only when none is recorded, or for `cordis:` builtins. Dependency analysis considers every recorded module. Entries sharing a plugin runtime use one replacement operation, while each Loader entry selects its replacement by its original namespace. Module imports and export normalization finish before the old runtime is removed. Failed imports or activation restore the previous modules and plugin implementations and clean up partially activated replacements.
An instance without a Loader entry has no recorded module identity. If a shared runtime's replacement modules have different plugin callbacks, HMR reports an ambiguity error and rolls back instead of assigning that instance to an arbitrary module.
Package invalidation lives in `packages/boot/hmr/src/package-manifest.ts`, which encapsulates its Node internal interfaces. `index.ts` dispatches manifest changes and locates loaded entries by module identity. HMR's `node_modules` exclusion is unchanged.
| Interface | Action |
|---|---|
| modules binding `readPackageJSON`, `getPackageScopeConfig`, `getPackageType` | Replaced: paths in an expired directory read the current file; other calls reach the native method |
| `package_json_reader.getNearestParentPackageJSON` | Replaced: expired directories bypass its JS cache; a lookup without a manifest returns the native absent result |
| The ESM Loader's `ResolveCache` instance | The prototype `get` is replaced for one lookup to obtain the instance and restored at once; the resolution cache is then cleared |
| CommonJS `Module._pathCache` | The request-to-filename cache is cleared |
| CommonJS `Module._load` | Default-resolver requests load by resolved filename; builtins and registered resolve hooks keep the original path |
The binding's `getNearestParentPackageJSON` is called only by `package_json_reader` and is not replaced. Each HMR instance owns a separate configuration cache. Its replacements are installed at the first expiry and restored when the service is disposed, alongside watcher and reload-queue cleanup.
### Future Work
- Online replacement inside `node_modules`, same-path reinstall, changed link targets, and cross-package reload propagation remain unsupported. Package updates need process restart; this does not guarantee that every management result already reports that requirement correctly.
- TSX versions using an asynchronous loader thread keep that thread's package configuration outside these hooks. Synchronizing it is deferred; this change does not provide general Worker cache synchronization.
- CommonJS private-request-cache refresh with registered synchronous resolve hooks is deferred. Those requests retain their original loader behavior.
- Disposing HMR restores the native readers, whose previous cached configuration can become visible again. Preserving invalidation state across HMR replacement is deferred.
## Alternatives considered
**Let HMR reload packages inside `node_modules`.** It requires finding affected plugins over a module graph that includes `node_modules`, preventing duplicate evaluation of shared libraries and Cordis, and handling missing dynamic-import edges, module side effects, and Workers. It is not done here; those scenarios keep requiring restart.
**Reload every plugin for a package.json change.** Configuration-only manifests do not require module evaluation. Manifests actually imported as JSON modules retain ordinary dependency-driven reloads.
**Re-import reloaded plugins by package name.** It would let an entry rename follow a source reload, but changes `partialReload`'s rule that the loaded URL is the reload unit. Entry renames already take effect when the Loader entry restarts, so this is not changed.
**Keep one namespace per entry name.** Entries with the same name and base URL can have loaded different modules before and after a manifest change. Keeping only the first or last namespace loses a live module; an uninitialized entry must not erase another entry's imported namespace.
## Verification
| Coverage | Location |
|---|---|
| Expired exports, main, imports, type, scope, and nearest manifests; native-reader parity; actual CommonJS loads and retained module instances; the `node_modules` boundary; restoration | `packages/boot/hmr/tests/package-manifest.spec.ts` |
| Configuration-only manifests, JSON-module and host reloads, source reloads, same-name and shared-runtime entries, import and activation rollback, entry restarts, and the `node_modules` exclusion | `packages/boot/hmr/tests/package-manifest-dispatch.spec.ts` |
Tests need no API key and make no model calls.
## Consequences
- In the supported thread, package.json configuration reads use the changed fields; JSON modules retain their own reload behavior. The separate module, loader-thread, and Worker limitations above still apply.
- Package configuration below expired directories is parsed in JavaScript and must keep the native reader's field and error semantics; tests compare it with the native reader on Node 22, 24, and 26.
- The implementation depends on several Node internal interfaces, and these tests must run again for Node upgrades.
@@ -0,0 +1,78 @@
# Agent Note: HMR 包配置失效
Status: implemented
[English](2026-09-30-profile-package-refresh-and-manifest-invalidation.md) | 中文
## Problem
- **包配置与模块求值使用独立缓存。** 清理插件的模块缓存不会刷新 Node 用于 exports、main、imports 和格式判断的 package.json 字段。package.json 也可以被作为 JSON 模块导入,此时其消费方需要普通的模块重载。
## Decision
### 职责
| 负责方 | 做什么 | 不做什么 |
|---|---|---|
| HMR(`packages/boot/hmr`) | 使包配置失效,并保留作为 JSON 模块加载的 manifest 的普通重载 | 不因仅作为配置的 manifest 重载插件;不处理 `node_modules` 包替换 |
### HMR 包配置失效
HMR 的文件变化分派中,文件名为 `package.json` 且不在 `node_modules` 内的变化,交给 `PackageManifests.invalidate()`。位于宿主依赖图中的 manifest 仍请求宿主重载;作为 JSON 模块加载的 manifest 仍重载消费方。仅作为配置的 manifest 不安排模块重载。配置专属路径仍由其独立 watcher 处理。同一批的源码变化随后重载。
`invalidate(manifest)` 把该目录登记为已失效,之后:
- 位于该目录下的 package.json 读取、scope 查询、type 查询、最近 package.json 查询,都按当前磁盘解析。消费方经由链接路径访问时,按真实目录判断归属。
- 清空 ESM `ResolveCache`。其条目不记录查询过的全部 manifest,包入口也可能解析到包目录之外。
- 基于相同原因清空 CommonJS `_pathCache`。无关请求重新计算解析结果,不卸载其模块。
- 使用默认 resolver 的 CommonJS 加载把重新解析的文件名交给原生 loader。其私有请求别名不再选中旧入口,已缓存的模块实例保持不变。
仅使配置失效不会改变已加载模块。Loader entry 保留导出规范化之前的原始导入结果。HMR 将已导入的 Node 模块对象与缓存中 ModuleJob 的模块命名空间按对象身份匹配,重载已加载 URL。所有匹配的 entry 记录,包括已停用的 entry,仅在重载成功后更新,失败时保持不变。停止并重新启动 Loader entry 时,会重新按包名解析并选择新入口。
entry 名称以配置树 base URL 为作用域,并保留所有不同的已导入命名空间。尚未初始化的 entry 只登记名称,不抹去已加载的命名空间;仅在没有记录命名空间或入口为 `cordis:` 内置模块时按名称解析。依赖分析考虑每个已记录模块。共享插件运行时的 entry 使用同一次替换操作,但每个 Loader entry 按自身原命名空间选择替换实现。模块导入与导出规范化在移除旧运行时之前完成。导入或激活失败时,恢复原来的模块和插件实现,并清理已部分激活的替换实例。
没有 Loader entry 的实例不具备已记录的模块身份。如果共享运行时的替换模块具有不同的插件回调,HMR 会报告歧义错误并回滚,而不是将该实例随意分配给某个模块。
包配置失效实现在 `packages/boot/hmr/src/package-manifest.ts`,封装其使用的 Node internal 接口。`index.ts` 分派 manifest 变更,并按模块身份定位已加载 entry。HMR 的 `node_modules` 排除规则不变。
| 接口 | 动作 |
|---|---|
| modules binding 的 `readPackageJSON`、`getPackageScopeConfig`、`getPackageType` | 替换:路径位于已失效目录时按当前磁盘解析,其余调用原生方法 |
| `package_json_reader.getNearestParentPackageJSON` | 替换:已失效目录绕过它的 JS 缓存;没有 manifest 时返回原生的缺失结果 |
| ESM Loader 的 `ResolveCache` 实例 | 临时替换原型的 `get`,取得实例后立即恢复,再清空解析缓存 |
| CJS `Module._pathCache` | 清空请求到文件名的缓存 |
| CJS `Module._load` | 默认 resolver 的请求按解析后的文件名加载;内置模块及注册的 resolve hooks 保留原路径 |
binding 的 `getNearestParentPackageJSON` 只被 `package_json_reader` 自己调用,不替换。每个 HMR 实例拥有独立的配置缓存,在第一次失效时安装 hook,并在服务销毁时恢复,同时清理 watcher 和重载队列。
### 后续工作
- 不支持 `node_modules` 内在线替换、同路径重装、link 换目标及跨包重载传播。包更新需要重启进程;这不保证每个管理操作的结果都已正确报告这一要求。
- 使用异步 loader 线程的 TSX 版本,其线程内包配置不受这些 hook 管理。线程同步留待后续;本次不提供通用 Worker 缓存同步。
- 注册了同步 resolve hooks 时,CommonJS 私有请求缓存刷新留待后续;这些请求保留原有 loader 行为。
- 销毁 HMR 会恢复原生 reader,其此前缓存的配置可能重新可见。跨 HMR 替换保留失效状态留待后续。
## Alternatives considered
**让 HMR 支持 `node_modules` 内的包重载。** 这需要沿包含 `node_modules` 的模块图确定受影响插件,防止共享库和 Cordis 被重复求值,还要处理动态 import 漏边、模块副作用和 Worker。本次不做,这些场景继续要求重启。
**package.json 变化时重载所有插件。** 仅作为配置的 manifest 不要求模块求值。实际作为 JSON 模块导入的 manifest 保留普通的依赖驱动重载。
**按插件名重新 import 被重载的插件。** 这能让入口改名随源码重载生效,但会改变 `partialReload` 以已加载 URL 为重载单位的规则。入口改名在 Loader entry 重启时已经生效,所以本次不改。
**每个 entry 名称只保留一个命名空间。** 名称和 base URL 相同的 entry,可能在 manifest 变化前后加载了不同模块。只保留最早或最后一个命名空间会遗漏仍在使用的模块;尚未初始化的 entry 也不能抹去其他 entry 的已导入命名空间。
## Verification
| 覆盖面 | 位置 |
|---|---|
| 失效后的 exports、main、imports、type、scope、最近 package.json,原生 reader 对照,实际 CommonJS 加载及旧模块实例保留,node_modules 边界,恢复 | `packages/boot/hmr/tests/package-manifest.spec.ts` |
| 配置专用 manifest、JSON 模块与宿主重载、源码重载、同名及共享运行时的 entry、导入与激活失败回滚、entry 重启和 node_modules 排除规则 | `packages/boot/hmr/tests/package-manifest-dispatch.spec.ts` |
测试不需要 API key,不调用模型。
## Consequences
- 在支持的线程内,package.json 配置读取使用更新后的字段;JSON 模块保留自己的重载行为。上述模块、loader 线程及 Worker 的独立限制仍然适用。
- 包配置读取改由 JS 解析 package.json,需要与原生读取保持字段和错误语义一致;测试在 Node 22、24、26 上与原生读取逐项比较。
- 依赖多个 Node internal 接口,Node 升级时需要重跑这些测试。
+5 -1
View File
@@ -29,7 +29,11 @@ export function installOfficeEngineResolution(runtimeDir: string): ModuleHooks |
return registerHooks({
resolve(specifier, context, nextResolve) {
const resolved = nextResolve(specifier, context)
if (!/^@deepseek-ai\/libreoffice-kit-(?:darwin|win32|linux)-/u.test(specifier)) return resolved
if (!resolved.url.startsWith('file:')) return resolved
const engineRequest = /^@deepseek-ai\/libreoffice-kit-(?:darwin|win32|linux)-/u.test(specifier)
const engineTarget = /\/node_modules\/@deepseek-ai\/libreoffice-kit-(?:darwin|win32|linux)-[^/]+\//u
.test(new URL(resolved.url).pathname)
if (!engineRequest && !engineTarget) return resolved
const canonical = pathToFileURL(realpathSync(fileURLToPath(resolved.url))).href
if (!canonical.startsWith(source)) {
if (canonical.startsWith(pathToFileURL(archive + '/').href)) {
+48 -2
View File
@@ -1,19 +1,26 @@
import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
import { createRequire, type ModuleHooks } from 'node:module'
import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { dirname, join, sep } from 'node:path'
import { afterEach, expect, it } from 'vitest'
import { installOfficeEngineResolution } from '../src/office-engine.ts'
import { PackageManifests } from '../../../packages/boot/hmr/src/package-manifest.ts'
const roots: string[] = []
const hooks: ModuleHooks[] = []
const disposers: Array<() => void> = []
afterEach(() => {
for (const dispose of disposers.splice(0).reverse()) dispose()
for (const hook of hooks.splice(0)) hook.deregister()
const cache = createRequire(import.meta.url).cache
for (const path of Object.keys(cache)) {
if (roots.some(root => path.startsWith(root + sep))) Reflect.deleteProperty(cache, path)
}
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true })
})
function fixture(runtimeName = 'dsh') {
const root = mkdtempSync(join(tmpdir(), 'desktop-office-resolution-'))
const root = realpathSync(mkdtempSync(join(tmpdir(), 'desktop-office-resolution-')))
roots.push(root)
const runtime = join(root, 'app.asar', runtimeName)
const manifest = 'node_modules/@deepseek-ai/libreoffice-kit-darwin-arm64/package.json'
@@ -40,6 +47,45 @@ it('resolves engine manifests to physical directories and leaves unrelated modul
expect(f.require('@deepseek-ai/libreoffice-kit/package.json')).toEqual({ name: '@deepseek-ai/libreoffice-kit' })
})
it('resolves an absolute engine manifest to its unpacked copy', () => {
const f = fixture()
expect(f.require(join(f.runtime, f.manifest)))
.toMatchObject({ path: realpathSync(dirname(join(f.root, 'app.asar.unpacked', 'dsh', f.manifest))) })
})
it.each(['exports', 'main'] as const)('refreshes a cached plugin %s while Office resolution stays active', (field) => {
const f = fixture()
const packageRoot = join(f.root, 'plugin')
mkdirSync(packageRoot)
const manifest = join(packageRoot, 'package.json')
const a = join(packageRoot, 'a.cjs')
const b = join(packageRoot, 'b.cjs')
writeFileSync(manifest, JSON.stringify({ name: 'plugin', [field]: './a.cjs' }))
writeFileSync(a, 'module.exports = { marker: "a" }')
writeFileSync(b, 'module.exports = { marker: "b" }')
symlinkSync(packageRoot, join(f.runtime, 'node_modules', 'plugin'), process.platform === 'win32' ? 'junction' : 'dir')
const require = createRequire(join(f.runtime, 'package.json'))
const original: unknown = require('plugin')
const next: unknown = require(b)
const originalModule = require.cache[a]
const nextModule = require.cache[b]
const engine = f.require('@deepseek-ai/libreoffice-kit-darwin-arm64/package.json')
expect(original).toEqual({ marker: 'a' })
expect(next).toEqual({ marker: 'b' })
expect(require('plugin')).toBe(original)
const manifests = new PackageManifests()
disposers.push(() => { manifests.dispose() })
writeFileSync(manifest, JSON.stringify({ name: 'plugin', [field]: './b.cjs' }))
manifests.invalidate(manifest)
expect(require('plugin')).toBe(next)
expect(require.resolve('plugin')).toBe(b)
expect(require.cache[a]).toBe(originalModule)
expect(require.cache[b]).toBe(nextModule)
expect(require(a)).toBe(original)
expect(f.require('@deepseek-ai/libreoffice-kit-darwin-arm64/package.json')).toBe(engine)
expect(f.require(join(f.runtime, f.manifest))).toBe(engine)
})
it('rejects an engine missing from the unpacked tree instead of using its archived copy', () => {
const f = fixture()
rmSync(join(f.root, 'app.asar.unpacked'), { recursive: true })
+1 -1
View File
@@ -1305,7 +1305,7 @@ export interface Config {
## `@deepseek-ai/dsh-hmr`
- `refs`: `ChokidarOptions` (`chokidar`)
- `source`: [`packages/boot/hmr/src/index.ts:51`](../packages/boot/hmr/src/index.ts)
- `source`: [`packages/boot/hmr/src/index.ts:53`](../packages/boot/hmr/src/index.ts)
```ts config-catalog
/** Module roots and watcher timing, with Chokidar deployment options. */
+1 -1
View File
@@ -1307,7 +1307,7 @@ export interface Config {
## `@deepseek-ai/dsh-hmr`
- `refs`: `ChokidarOptions` (`chokidar`)
- `source`: [`packages/boot/hmr/src/index.ts:51`](../packages/boot/hmr/src/index.ts)
- `source`: [`packages/boot/hmr/src/index.ts:53`](../packages/boot/hmr/src/index.ts)
```ts config-catalog
/** Module roots and watcher timing, with Chokidar deployment options. */
+2 -2
View File
@@ -51,8 +51,8 @@ This matrix shows which packages dispatch each harness-owned event and which pac
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:59`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-observation-policy`](../packages/fs/fs-observation-policy) |
| `goal/activation-changed` | `emit` | [`packages/goal/goal/src/types.ts:150`](../packages/goal/goal/src/types.ts) | [`goal`](../packages/goal/goal) (`emit`) | `remotes` |
| `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-round-driver`](../packages/goal/goal-round-driver) |
| `hmr/change` | `emit` | [`packages/boot/hmr/src/index.ts:30`](../packages/boot/hmr/src/index.ts) | [`hmr`](../packages/boot/hmr) (`emit`) | - |
| `hmr/reload` | `emit` | [`packages/boot/hmr/src/index.ts:35`](../packages/boot/hmr/src/index.ts) | [`hmr`](../packages/boot/hmr) (`emit`) | - |
| `hmr/change` | `emit` | [`packages/boot/hmr/src/index.ts:32`](../packages/boot/hmr/src/index.ts) | [`hmr`](../packages/boot/hmr) (`emit`) | - |
| `hmr/reload` | `emit` | [`packages/boot/hmr/src/index.ts:37`](../packages/boot/hmr/src/index.ts) | [`hmr`](../packages/boot/hmr) (`emit`) | - |
| `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | [`acp`](../packages/acp/acp), `remotes` |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:75`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
| `permission-presets/catalog-changed` | `emit` | [`packages/interaction/permission-presets/src/types.ts:48`](../packages/interaction/permission-presets/src/types.ts) | [`permission-presets`](../packages/interaction/permission-presets) (`events.dispatch`) | `remotes` |
+2 -2
View File
@@ -53,8 +53,8 @@
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:59`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-observation-policy`](../packages/fs/fs-observation-policy) |
| `goal/activation-changed` | `emit` | [`packages/goal/goal/src/types.ts:150`](../packages/goal/goal/src/types.ts) | [`goal`](../packages/goal/goal) (`emit`) | `remotes` |
| `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-round-driver`](../packages/goal/goal-round-driver) |
| `hmr/change` | `emit` | [`packages/boot/hmr/src/index.ts:30`](../packages/boot/hmr/src/index.ts) | [`hmr`](../packages/boot/hmr) (`emit`) | - |
| `hmr/reload` | `emit` | [`packages/boot/hmr/src/index.ts:35`](../packages/boot/hmr/src/index.ts) | [`hmr`](../packages/boot/hmr) (`emit`) | - |
| `hmr/change` | `emit` | [`packages/boot/hmr/src/index.ts:32`](../packages/boot/hmr/src/index.ts) | [`hmr`](../packages/boot/hmr) (`emit`) | - |
| `hmr/reload` | `emit` | [`packages/boot/hmr/src/index.ts:37`](../packages/boot/hmr/src/index.ts) | [`hmr`](../packages/boot/hmr) (`emit`) | - |
| `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | [`acp`](../packages/acp/acp), `remotes` |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:75`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
| `permission-presets/catalog-changed` | `emit` | [`packages/interaction/permission-presets/src/types.ts:48`](../packages/interaction/permission-presets/src/types.ts) | [`permission-presets`](../packages/interaction/permission-presets) (`events.dispatch`) | `remotes` |
+4 -4
View File
@@ -21,8 +21,8 @@
en: 45ee194ad781d603
zh: b3d7fcd8ed73717c
/deepseek-ai-dsh-hmr/understand-the-implementation:
en: 247499831fdee81f
zh: 690f1859c1f73bbe
en: 227c3c99c3c4b597
zh: 853b656340cb8bce
/deepseek-ai-dsh-hmr/model-experience:
en: 215e7ba838619b7b
zh: a311da3843709f90
@@ -39,8 +39,8 @@
en: 89e0624fafd701fb
zh: 19d36a33c1d9af9d
/deepseek-ai-dsh-hmr/known-limitations-and-deferred-work:
en: 379bc2c72fd9e53e
zh: 1fff1e0e676bb161
en: 0f46d4df1dab7f3b
zh: d172835159a20cce
/deepseek-ai-dsh-hmr/known-limitations-and-deferred-work/dev-note:
en: 7972e4b2005ec9a4
zh: 754c49ec538c164c
+14 -2
View File
@@ -56,7 +56,15 @@ Chokidar options, including polling, retain their existing meaning. Exact config
`watchConfig()` registers an awaited configuration handler. `runExclusive()` serializes configuration changes and Loader updates with automatic reloads and rejects nested transactions. Package installation and removal run outside this queue. HMR does not acquire the package writer lock; manifest notifications reload only when the ordered `dsh.profile.bundles` list changes. Profile and home patch changes also trigger recomposition. File events received during a configuration transaction are processed afterward. Include refreshes and profile reconciliation reach plugins through ordinary Loader entry updates; Loader commits volatile-only changes in place.
App-boot owns profile parsing and patch precedence. HMR reads the launcher’s data-only `profileContext`, registers the profile manifest and both user patch watches during initialization, and waits for application readiness before processing changes. Its disposal closes the watchers and cancels reloads waiting for startup. HMR also owns module-cache replacement and reload scheduling. Configuration watchers start outside the active transaction context so later notifications can enter the queue.
App-boot owns profile parsing and patch precedence. HMR reads the launcher’s data-only `profileContext`, registers the profile manifest and both user patch watches during initialization, and waits for application readiness before processing changes. Its disposal closes the watchers and cancels reloads waiting for startup. Configuration watchers start outside the active transaction context so later notifications can enter the queue.
A changed `package.json` outside `node_modules` expires that package's cached configuration. A configuration-only manifest triggers no module reload; a manifest imported as a JSON module retains dependency-driven reloads, and one in the host dependency graph retains host reloads. Dedicated configuration watches keep their existing ownership. A new package entry takes effect when its Loader entry restarts.
Loader entries retain their raw import results. HMR retains every distinct imported namespace for each entry name and configuration-tree base URL; an uninitialized entry does not overwrite another entry's namespace. HMR matches those objects to cached ModuleJob namespaces instead of resolving entry names again. Names without an imported namespace and `cordis:` builtins keep name-based resolution. Source reloads therefore use the loaded modules after `exports` or `main` changes.
Entries sharing a runtime are replaced together, with each entry receiving the implementation for its own module. Duplicate entries for one module retain their separate plugin instances. On replacement failure, HMR removes partially activated replacements and restores the previous modules and plugin implementations. A successful reload updates every matching entry record, including disabled entries; a failed replacement leaves those records unchanged. Entries that have never loaded remain uninitialized.
[package-manifest.ts](src/package-manifest.ts) owns the Node internal interfaces for package reads, scope, type, and nearest-manifest lookups. Each HMR instance owns its configuration cache and restores its installed hooks on disposal. Manifest invalidation clears the ESM `ResolveCache` and CommonJS `_pathCache`: neither records every consulted manifest, and entries may point outside their package directory. CommonJS loads resolve requests with Node's default resolver before entering the native loader, bypassing old request aliases while preserving evaluated modules. Synchronous resolve hooks receive the resulting filenames. Ordinary module replacement remains HMR's separate operation.
Watched module paths use Node ESM resolution's `realpathSync()` spelling, including Windows short directory names, so file events match the module cache.
@@ -86,7 +94,11 @@ Reloading a contributing plugin can change later request prefixes; HMR does not
<a id="known-limitations-and-deferred-work"></a>
- Module replacement requires Node loader internals. Framework dependency changes call the host-provided `loader.exit()` hook; HMR itself does not restart the process.
- Replacing installed package versions still requires a restart through Plugin Manager. The browser Client module graph retains its separate browser-side loading mechanism.
- Replacing installed package versions still requires a restart through Plugin Manager; manifests below `node_modules` keep Node's cached configuration. The browser Client module graph retains its separate browser-side loading mechanism.
- TSX loaders and ordinary application-created Workers are outside the supported package-cache refresh scope.
- CommonJS requests must be resolvable by Node's default resolver. Synchronous hooks may postprocess the resulting filenames, as Desktop's Office resolver does; hooks requiring the original request name or introducing virtual requests are unsupported. Builtins retain Node's original loading behavior.
- Future work: retain package invalidation across HMR disposal and replacement. Restoring the native readers can expose their older cached configuration again.
- If a shared runtime's replacement modules have different plugin callbacks, an instance without a Loader entry cannot be assigned to one of them. HMR reports an ambiguity error and rolls back the reload.
- `watchConfig()` resolves when Chokidar reports readiness. On darwin, libuv starts the FSEvents stream afterwards on its own thread, so a write that lands within milliseconds of registration is not reported until the next event in that directory; edits made after startup are unaffected.
### Dev Note
+14 -2
View File
@@ -56,7 +56,15 @@ Chokidar 选项(包括轮询)保持原有含义。精确配置监听同时
`watchConfig()` 注册会被等待的配置处理器。`runExclusive()` 将配置变更、Loader 更新与自动重载串行化,并拒绝嵌套事务。包安装和删除在该队列之外执行。HMR 不获取包操作写锁;manifest 通知仅在有序的 `dsh.profile.bundles` 列表变化时触发重载。profile 与 home patch 变化也会触发重新组合。配置事务期间收到的文件事件在事务结束后处理。 Include 刷新和 profile 重载都通过普通的 Loader 条目更新到达插件;仅 volatile 变化由 Loader 就地提交。
App-boot 负责 profile 解析和 patch 优先级规则。HMR 读取启动器提供的纯数据 `profileContext`,在初始化时注册 profile manifest 和两份用户 patch 的监听,并等待应用就绪后处理更改。销毁 HMR 时会关闭监听器并取消等待启动的重载。HMR 也负责模块缓存替换和重载调度。配置监听器在当前事务上下文之外启动,使后续通知可以进入队列。
App-boot 负责 profile 解析和 patch 优先级规则。HMR 读取启动器提供的纯数据 `profileContext`,在初始化时注册 profile manifest 和两份用户 patch 的监听,并等待应用就绪后处理更改。销毁 HMR 时会关闭监听器并取消等待启动的重载。配置监听器在当前事务上下文之外启动,使后续通知可以进入队列。
`node_modules` 之外的 `package.json` 变化使该包的配置缓存失效。仅作为配置的 manifest 不触发模块重载;作为 JSON 模块导入的 manifest 保留依赖驱动重载,位于宿主依赖图中的 manifest 保留宿主重载。专属配置监听保留原有归属。新包入口在 Loader entry 重启时生效。
Loader entry 保留原始导入结果。HMR 为每个 entry 名称与配置树 base URL 保留所有不同的已导入命名空间;尚未初始化的 entry 不会覆盖其他 entry 的命名空间。HMR 将这些对象与缓存中 ModuleJob 的模块命名空间按对象身份匹配,不再重新解析 entry 名称。没有已导入命名空间的名称和 `cordis:` 内置入口仍按名称解析。因此 `exports` 或 `main` 改变后,源码重载仍使用已加载模块。
共享同一运行时的 entry 一起替换,每个 entry 使用其自身模块对应的实现。同一模块的重复 entry 仍保有各自的插件实例。替换失败时,HMR 清理已部分激活的替换实例,并恢复原来的模块和插件实现。重载成功后更新所有匹配的 entry 记录,包括已停用的 entry;替换失败时这些记录保持不变。从未加载过的 entry 仍保持未初始化状态。
[package-manifest.ts](src/package-manifest.ts) 封装 package 读取、scope、type 和最近 manifest 查询所需的 Node internal 接口。每个 HMR 实例拥有自己的配置缓存,销毁时恢复它安装的 hook。manifest 失效会清空 ESM `ResolveCache` 和 CommonJS `_pathCache`:它们都不记录查询过的全部 manifest,入口也可能指向包目录之外。CommonJS 加载先用 Node 默认 resolver 解析请求,再进入原生 loader,绕过旧请求别名并保留已求值模块。同步 resolve hook 收到的是解析后的文件名。普通模块替换仍由 HMR 独立处理。
被监听模块的路径沿用 Node ESM 解析所用的 `realpathSync()` 表示,包括 Windows 短目录名,使文件事件与模块缓存匹配。
@@ -86,7 +94,11 @@ App-boot 负责 profile 解析和 patch 优先级规则。HMR 读取启动器提
<a id="known-limitations-and-deferred-work"></a>
- 模块替换需要 Node loader 内部接口。框架依赖变化调用宿主提供的 `loader.exit()` 钩子;HMR 本身不重启进程。
- 通过插件管理器替换已安装包版本仍需要重启。浏览器 Client 模块图保留独立的浏览器侧加载机制。
- 通过插件管理器替换已安装包版本仍需要重启;`node_modules` 之下的 manifest 保留 Node 缓存的配置。浏览器 Client 模块图保留独立的浏览器侧加载机制。
- TSX loader 和普通应用自行创建的 Worker 不在包缓存刷新支持范围内。
- CommonJS 请求必须能由 Node 默认 resolver 解析。同步 hook 可以像 Desktop 的 Office resolver 一样处理解析后的文件名;依赖原始请求名或引入虚拟请求的 hook 不受支持。内置模块保留 Node 原有的加载行为。
- 后续工作:跨 HMR 销毁和替换保留包配置失效状态。恢复原生 reader 后,其旧配置缓存可能重新可见。
- 如果共享运行时的替换模块具有不同的插件回调,没有 Loader entry 的实例就无法确定应使用哪一个。HMR 会报告歧义错误并回滚此次重载。
- `watchConfig()` 在 Chokidar 报告就绪时 resolve。darwin 上 libuv 随后才在自己的线程启动 FSEvents 流,因此注册后数毫秒内落地的写入要等到该目录的下一个事件才会被报告;启动之后的编辑不受影响。
### 开发备注
+1
View File
@@ -38,6 +38,7 @@
"dependencies": {
"@babel/code-frame": "^7.29.0",
"chokidar": "^4.0.3",
"node-addon-require-builtin": "^0.1.6",
"picomatch": "^4.0.3",
"@deepseek-ai/schemastery": "workspace:~"
},
+188 -38
View File
@@ -1,15 +1,17 @@
/** Serialized module and profile-configuration reloads. */
import assert from 'node:assert/strict'
import { AsyncLocalStorage } from 'node:async_hooks'
import { watchConfig as watchExactConfig } from './watch-config.ts'
import { Context, Inject, Service, type Plugin } from '@deepseek-ai/cordis'
import { ModuleLoader, type ModuleJob, type ResolveResult } from '@deepseek-ai/cordis-plugin-loader'
import { Context, Inject, Service, type Fiber, type Plugin } from '@deepseek-ai/cordis'
import type { ModuleLoader, ModuleJob, ResolveResult } from '@deepseek-ai/cordis-plugin-loader'
import type { Include } from '@deepseek-ai/cordis-plugin-include'
import { FSWatcher, watch, type ChokidarOptions } from 'chokidar'
import { basename, dirname, join, relative, resolve } from 'node:path'
import { basename, dirname, join, relative, resolve, sep } from 'node:path'
import { readFileSync, realpathSync } from 'node:fs'
import { readProfileManifest, readProfilePatches, reconcileProfilePatches, PROFILE_PATCH_FILENAME } from '@deepseek-ai/dsh-app-boot'
import type {} from '@deepseek-ai/dsh-cmdline'
import { handleError } from './error.ts'
import { PackageManifests } from './package-manifest.ts'
import type {} from '@deepseek-ai/cordis-plugin-timer'
import { fileURLToPath, pathToFileURL } from 'node:url'
import { createRequire } from 'node:module'
@@ -79,9 +81,129 @@ async function loadDependencies(job: ModuleJob, ignored = new Set<string>()) {
/** Module location and runtime retained during a replacement. */
export interface Reload {
filename: string
/** Original namespaces and URLs of all entry modules participating in this runtime replacement. */
modules: ReloadModules
runtime?: Plugin.Runtime | undefined
}
/** Entry names with distinct namespaces; iteration falls back to undefined only for names without a loaded namespace. */
class EntryNamespaces {
private readonly names = new Map<string, Set<unknown>>()
constructor(private internal: ModuleLoader, private moduleUrls: ReadonlyMap<unknown, string>) {}
static getModuleUrls(internal: ModuleLoader): Map<unknown, string> {
const moduleUrls = new Map<unknown, string>()
for (const url of internal.loadCache.keys()) {
const job = internal.loadCache.get(url)
if (!job?.module) continue
try {
moduleUrls.set(job.module.getNamespace(), url)
} catch (_error) {
// Node also caches unfinished jobs whose namespace is not available yet.
}
}
return moduleUrls
}
add(name: string, moduleNamespace: unknown): this {
const namespaces = this.names.get(name) ?? new Set<unknown>()
namespaces.add(moduleNamespace)
this.names.set(name, namespaces)
return this
}
*[Symbol.iterator](): Generator<[name: string, moduleNamespace: unknown]> {
for (const [name, namespaces] of this.names) {
for (const moduleNamespace of namespaces) {
if (moduleNamespace === undefined && namespaces.size > 1) continue
yield [name, moduleNamespace]
}
}
}
async resolve(name: string, baseUrl: string, moduleNamespace: unknown): Promise<Pick<ResolveResult, 'url'>> {
if (moduleNamespace === undefined || name.startsWith('cordis:')) {
switch (this.internal.version) {
case 'v1': return await this.internal.resolve(name, baseUrl, {})
case 'v2': return this.internal.resolveSync(baseUrl, { specifier: name, attributes: {} })
}
}
const url = this.moduleUrls.get(moduleNamespace)
if (url === undefined) throw new Error(`HMR cannot locate the loaded module for ${name} from ${baseUrl}`)
return { url }
}
}
type ReloadFiber = {
fiber: Fiber
entry: Fiber['entry']
config: unknown
moduleNamespace: unknown
}
type ReloadModule = { filename: string; moduleNamespace: unknown; plugin: Plugin }
/** Entry modules belonging to one runtime, keyed by their original namespaces across replacement. */
class ReloadModules {
private readonly modules = new Map<unknown, ReloadModule>()
static include(reloads: ReadonlyMap<Plugin, Reload>, plugin: Plugin, job: ModuleJob, runtime: Plugin.Runtime | undefined): boolean {
let info = reloads.get(plugin)
if (info === undefined && runtime) info = [...reloads.values()].find(info => info.runtime === runtime)
if (info === undefined) return false
info.modules.add(job, plugin)
return true
}
add(job: ModuleJob, plugin: Plugin): this {
assert(job.module, `HMR pending module is missing: ${job.url}`)
const moduleNamespace: unknown = job.module.getNamespace()
this.modules.set(moduleNamespace, { filename: job.url, moduleNamespace, plugin })
return this
}
async importRemaining(loader: Context['loader'], getOuterStack: () => string[], primary: ReloadModule): Promise<ReloadModules> {
const replacements = new ReloadModules()
for (const [originalNamespace, { filename }] of this.modules) {
if (filename === primary.filename) {
replacements.modules.set(originalNamespace, primary)
continue
}
const moduleNamespace: unknown = await loader.import(filename, getOuterStack)
const plugin = loader.unwrapExports(moduleNamespace) as Plugin
replacements.modules.set(originalNamespace, { filename, moduleNamespace, plugin })
}
return replacements
}
*getActiveImplementations(ctx: Context, plugin: Plugin, fibers: readonly ReloadFiber[]): Generator<[ReloadFiber, Plugin]> {
const callbacks = new Set([...this.modules.values()].map(value => ctx.registry.resolve(value.plugin)))
const implementations = fibers.map((previousFiber) => {
if (previousFiber.entry === undefined) return [previousFiber, plugin] as const
const replacement = this.modules.get(previousFiber.moduleNamespace)
const implementation = replacement ? replacement.plugin : ctx.loader.unwrapExports(previousFiber.moduleNamespace) as Plugin
callbacks.add(ctx.registry.resolve(implementation))
return [previousFiber, implementation] as const
})
for (const [previousFiber, implementation] of implementations) {
if (previousFiber.fiber.parent.fiber.uid === null) continue
if (previousFiber.entry === undefined && callbacks.size > 1) {
throw new Error('HMR replacement is ambiguous for a plugin instance without a Loader entry')
}
yield [previousFiber, implementation]
}
}
updateEntries(loader: Context['loader']): void {
for (const [originalNamespace, { moduleNamespace }] of this.modules) {
for (const entry of loader.entries()) {
if (entry.moduleNamespace === originalNamespace) entry.moduleNamespace = moduleNamespace
}
}
}
}
/** Hot reload service with Cordis-compatible module configuration and events. */
@Inject('loader')
@Inject('timer')
@@ -131,6 +253,7 @@ class Hmr extends Service {
private applicationReady: Promise<boolean> = Promise.resolve(true)
private closing = false
private readonly configPaths = new Set<string>()
private readonly manifests = new PackageManifests()
/** Serialize a caller-owned mutation with all automatic reload paths.
* @param operation Work that must not overlap module or configuration replacement.
@@ -185,22 +308,13 @@ class Hmr extends Service {
this.baseDir = fileURLToPath(new URL(config.base || '.', ctx.baseUrl))
}
/**
* Resolve a module specifier to a URL, compatible with Node 22-24.
*/
private async _resolve(specifier: string, parentURL: string, attrs: ImportAttributes): Promise<ResolveResult> {
switch (this.internal.version) {
case 'v1': return await this.internal.resolve(specifier, parentURL, attrs)
case 'v2': return this.internal.resolveSync(parentURL, { specifier, attributes: attrs })
}
}
async* [Service.init](): AsyncGenerator<() => Promise<void>, void, unknown> {
yield async () => {
this.closing = true
await this.watcher?.close()
// A configuration reload may remove its own HMR entry.
if (!this.executing.getStore()) await this.operations
this.manifests.dispose()
}
const profile = this.ownerContext.get('profileContext')
@@ -276,15 +390,20 @@ class Hmr extends Service {
const filename = canonicalPath(resolve(watchBaseDir, path))
const configuredFilename = resolve(this.baseDir, path)
if (this.configPaths.has(filename) || this.configPaths.has(configuredFilename)) continue
const isManifest = basename(filename) === 'package.json' && !filename.includes(`${sep}node_modules${sep}`)
if (isManifest) {
this.manifests.invalidate(filename)
}
const url = pathToFileURL(filename).href
if (this.externals.has(url)) {
fullReload = true
continue
}
if (this.internal.loadCache.has(url)) {
if (this.internal.loadCache.has(url) || this.internal.loadCache.has(url, 'json')) {
this.stashed.add(url)
continue
}
if (isManifest) continue
const include = [...loader.entries()].map(entry => entry.subtree as Include | undefined)
.find(tree => tree?.filename === filename || tree?.filename === configuredFilename)
if (include !== undefined) includes.add(include)
@@ -408,23 +527,24 @@ class Hmr extends Service {
const pending = new Map<ModuleJob, Plugin>()
const reloads = new Map<Plugin, Reload>()
const moduleUrls = EntryNamespaces.getModuleUrls(this.internal)
// Build a map of plugin names per config tree URL.
// Plugin entry files are treated as atomic reload units.
const nameMap = new Map<string, Set<string>>()
const nameMap = new Map<string, EntryNamespaces>()
for (const entry of this.ctx.loader.entries()) {
const baseUrl = entry.parent.tree.ctx.baseUrl
if (baseUrl === undefined) throw new Error('HMR entry tree has no base URL')
const names = nameMap.get(baseUrl) ?? new Set<string>()
names.add(entry.options.name)
const names = nameMap.get(baseUrl) ?? new EntryNamespaces(this.internal, moduleUrls)
names.add(entry.options.name, entry.moduleNamespace)
nameMap.set(baseUrl, names)
}
// Resolve each plugin name to its file URL and check if it needs reload
// Find each plugin's loaded URL and check if it needs reload.
for (const [baseUrl, names] of nameMap) {
for (const name of names) {
for (const [name, moduleNamespace] of names) {
try {
const { url } = await this._resolve(name, baseUrl, {})
const { url } = await names.resolve(name, baseUrl, moduleNamespace)
if (this.declined.has(url)) continue
const job = this.internal.loadCache.get(url)
const plugin = this.ctx.loader.unwrapExports(job?.module?.getNamespace()) as Plugin | undefined
@@ -443,15 +563,27 @@ class Hmr extends Service {
const dependencies = [...await loadDependencies(job, this.declined)]
this.declined.add(job.url)
if (dependencies.length === 0) {
pending.delete(job)
continue
}
if (!dependencies.some(dep => this.accepted.has(dep))) continue
dependencies.forEach(dep => this.accepted.add(dep))
const runtime = this.ctx.registry.get(plugin)
if (ReloadModules.include(reloads, plugin, job, runtime)) continue
reloads.set(plugin, {
filename: job.url,
runtime: this.ctx.registry.get(plugin),
runtime,
modules: new ReloadModules().add(job, plugin),
})
}
// Re-export roots of a replaced runtime must not retain bindings to its old modules.
for (const [job, plugin] of pending) {
if (ReloadModules.include(reloads, plugin, job, this.ctx.registry.get(plugin))) this.accepted.add(job.url)
}
/**
* Clear module caches for all accepted files before re-importing.
*
@@ -502,14 +634,23 @@ class Hmr extends Service {
fibers: [...info.runtime?.fibers ?? []].map((fiber) => {
const entry = fiber.entry?.fiber?.uid === fiber.uid ? fiber.entry : undefined
const config: unknown = entry === undefined ? fiber._config : entry.options.config
return { fiber, entry, config }
return { fiber, entry, config, moduleNamespace: entry?.moduleNamespace }
}),
}))
const attempts: Array<(typeof generations)[number] & { replacement: Plugin }> = []
type ReloadAttempt = (typeof generations)[number] & {
replacement: Plugin
replacements: ReloadModules
activated: Fiber[]
}
const attempts: ReloadAttempt[] = []
try {
for (const generation of generations) {
const replacement = this.ctx.loader.unwrapExports(await this.ctx.loader.import(generation.filename, this.getOuterStack)) as Plugin
attempts.push({ ...generation, replacement })
const moduleNamespace: unknown = await this.ctx.loader.import(generation.filename, this.getOuterStack)
const replacement = this.ctx.loader.unwrapExports(moduleNamespace) as Plugin
const replacements = await generation.modules.importRemaining(this.ctx.loader, this.getOuterStack, {
filename: generation.filename, moduleNamespace, plugin: replacement,
})
attempts.push({ ...generation, replacement, replacements, activated: [] })
}
} catch (e) {
handleError(this.ctx, e)
@@ -517,11 +658,14 @@ class Hmr extends Service {
throw e
}
const reload = async (plugin: Plugin, fibers: (typeof generations)[number]['fibers']) => {
const activated = []
for (const previousFiber of fibers) {
if (previousFiber.fiber.parent.fiber.uid === null) continue
const fiber = previousFiber.fiber.parent.registry.plugin(plugin, previousFiber.config, this.getOuterStack).ctx.fiber
const reload = async (
plugin: Plugin,
fibers: (typeof generations)[number]['fibers'],
replacements = new ReloadModules(),
activated: Fiber[] = [],
) => {
for (const [previousFiber, implementation] of replacements.getActiveImplementations(this.ctx, plugin, fibers)) {
const fiber = previousFiber.fiber.parent.registry.plugin(implementation, previousFiber.config, this.getOuterStack).ctx.fiber
if (previousFiber.entry !== undefined) {
fiber.entry = previousFiber.entry
previousFiber.entry.fiber = fiber
@@ -533,7 +677,7 @@ class Hmr extends Service {
const removed = new Set<Plugin>()
try {
for (const { previous: plugin, replacement, filename, runtime, fibers } of attempts) {
for (const { previous: plugin, replacement, filename, runtime, fibers, replacements, activated } of attempts) {
if (!runtime) continue
const path = relative(this.baseDir, fileURLToPath(filename))
@@ -547,7 +691,7 @@ class Hmr extends Service {
}
try {
await reload(replacement, fibers)
await reload(replacement, fibers, replacements, activated)
this.ctx.logger.info('reload plugin at %C', path)
} catch (err) {
this.ctx.logger.warn('failed to reload plugin at %C', path)
@@ -558,14 +702,17 @@ class Hmr extends Service {
} catch (error) {
// Restore caches and re-register old plugins after a replacement failure.
rollback()
for (const { previous: plugin, replacement, fibers } of attempts) {
for (const { previous: plugin, fibers, activated } of attempts) {
if (!removed.has(plugin)) continue
try {
const replacementRuntime = this.ctx.registry.get(replacement)
const replacementFibers = [...replacementRuntime?.fibers ?? []]
this.ctx.registry.delete(replacement)
// Failed startup errors remain on fibers after their teardown finishes.
await Promise.allSettled(replacementFibers.map(fiber => fiber.await()))
for (const { runtime } of activated) {
if (runtime && this.ctx.registry.get(runtime.callback) === runtime) {
const replacementFibers = [...runtime.fibers]
this.ctx.registry.delete(runtime.callback)
// Failed startup errors remain on fibers after their teardown finishes.
await Promise.allSettled(replacementFibers.map(fiber => fiber.await()))
}
}
await reload(plugin, fibers)
} catch (err) {
this.ctx.logger.warn(err)
@@ -575,6 +722,9 @@ class Hmr extends Service {
}
await this.ctx.loader.await()
for (const { replacements } of attempts) {
replacements.updateEntries(this.ctx.loader)
}
this.ctx.emit('hmr/reload', reloads)
this.stashed = new Set()
}
+330
View File
@@ -0,0 +1,330 @@
/** Expire Node's cached package configuration for package directories whose manifest changed. */
import { isUtf8 } from 'node:buffer'
import { readFileSync, realpathSync } from 'node:fs'
import { createRequire, isBuiltin } from 'node:module'
import { basename, dirname, join, sep, toNamespacedPath } from 'node:path'
import { fileURLToPath, pathToFileURL } from 'node:url'
/** Package fields Node's resolver consumes; exports and imports hold JSON values. */
interface PackageConfig {
name?: string
main?: string
type: string
exports?: unknown
imports?: unknown
}
/** Serialized fields returned by the native modules binding: name, main, type, imports, exports, manifest path. */
type PackageTuple = [string | undefined, string | undefined, string, string | undefined, string | undefined, string]
/**
* Methods of `internalBinding('modules')` that read package configuration through Node's native cache. Node's
* package reader, CommonJS loader, and ESM resolver and format detection look them up at each call.
*/
interface PackageBinding {
readPackageJSON(this: void, path: string, isEsm?: boolean, base?: string, specifier?: string): PackageTuple | undefined
getPackageScopeConfig(this: void, url: string): PackageTuple | string
getPackageType(this: void, url: string): string | undefined
}
/**
* `internal/modules/package_json_reader` export that keeps its own nearest-manifest cache; the CommonJS loader calls it
* through the module object, so it is the only caller of the binding's nearest-manifest lookup.
*/
interface PackageReader {
getNearestParentPackageJSON(this: void, path: string): { data: PackageConfig; exists: boolean; path: string } | undefined
}
/** Node's ESM resolution cache: a Map from parent URL to the serialized request keys resolved from it. */
interface ResolveCacheClass {
prototype: { get: (this: Map<string, unknown>, key: string, parent: string) => unknown }
}
interface CommonJsModule {
_pathCache: Record<string, string>
_resolveFilename(request: string, parent: NodeJS.Module | null | undefined, isMain?: boolean): string
_load(request: string, parent: NodeJS.Module | null | undefined, isMain?: boolean, ...options: unknown[]): unknown
}
interface NativeAccess {
binding: PackageBinding
reader: PackageReader
ResolveCache: ResolveCacheClass
cjs: CommonJsModule
esmLoader: { resolveSync(...args: unknown[]): unknown }
}
/** Snapshot own methods; each disposer restores its method only while its replacement is still installed. */
function createReplaceHelper<T extends { [K in keyof T]: (...args: never[]) => unknown }>(target: T) {
const rawImpl = { ...target }
return {
rawImpl,
replaceMethod<K extends keyof T>(key: K, replacement: T[K]): () => void {
target[key] = replacement
return () => { if (target[key] === replacement) target[key] = rawImpl[key] }
},
}
}
/** Same lookup as the vendored Loader's `requireInternal`: `--expose-internals` first, then the builtin addon. */
function requireInternal(id: string): unknown {
const require = createRequire(import.meta.url)
if (process.execArgv.includes('--expose-internals')) {
try {
return require(id)
} catch (_error) {
// Some internals, such as internal/bootstrap/realm, stay private under --expose-internals.
}
}
try {
return (require('node-addon-require-builtin') as { requireBuiltin(id: string): unknown }).requireBuiltin(id)
} catch (_error) {
// Without the addon, the caller reports the missing interface.
}
}
function loadNodeInternals(): NativeAccess {
const realm = requireInternal('internal/bootstrap/realm') as { internalBinding(id: 'modules'): PackageBinding }
const esm = requireInternal('internal/modules/esm/loader') as {
getOrInitializeCascadedLoader(): NativeAccess['esmLoader']
}
return {
binding: realm.internalBinding('modules'),
reader: requireInternal('internal/modules/package_json_reader') as PackageReader,
ResolveCache: (requireInternal('internal/modules/esm/module_map') as { ResolveCache: ResolveCacheClass }).ResolveCache,
cjs: (requireInternal('internal/modules/cjs/loader') as { Module: CommonJsModule }).Module,
esmLoader: esm.getOrInitializeCascadedLoader(),
}
}
/**
* Read the ESM loader's resolution cache by observing one lookup through its prototype.
* The prototype method is restored before this returns.
*/
function captureEsmResolveCache(native: NativeAccess, parentURL: string): Map<string, unknown> | undefined {
const prototype = native.ResolveCache.prototype
const original = prototype.get
const seen = new Set<Map<string, unknown>>()
prototype.get = function (this: Map<string, unknown>, key: string, parent: string): unknown {
seen.add(this)
return Reflect.apply(original, this, [key, parent])
}
try {
const loader = native.esmLoader
/* v8 ignore else -- CI coverage runs the Node 24 loader; the Node 22 matrix exercises the v1 signature */
if ('getOrCreateModuleJob' in loader) loader.resolveSync(parentURL, { specifier: 'node:path', attributes: {} })
else loader.resolveSync('node:path', parentURL, {})
} finally {
prototype.get = original
}
return [...seen][0]
}
interface Configuration {
data: PackageConfig
serialized: PackageTuple
}
interface ImportContext {
base: string
specifier: string | undefined
}
/** Package directories whose manifests are read from disk instead of Node's native cache. */
export class PackageManifests {
private readonly directories = new Set<string>()
private readonly configurations = new Map<string, Configuration | undefined>()
/** Real directory of each directory a lookup reached; linked consumers name a package by its link path. */
private readonly realDirectories = new Map<string, string>()
private native: NativeAccess | undefined
private rawBinding!: PackageBinding
private rawReader!: PackageReader
private rawCommonJs!: Pick<CommonJsModule, '_load'>
private readonly restorers: Array<() => void> = []
/**
* Expire cached configuration for one package directory. Loaded modules stay evaluated; later resolutions and
* format checks read the manifest on disk.
* @param manifest - absolute path of the changed `package.json`.
*/
invalidate(manifest: string): void {
const native = this.installPackageHooks()
const directory = dirname(manifest)
this.directories.add(directory)
this.configurations.clear()
this.realDirectories.clear()
// A package's entry may resolve outside its directory; cache keys do not record the manifests they consulted.
for (const key of Object.keys(native.cjs._pathCache)) Reflect.deleteProperty(native.cjs._pathCache, key)
const directoryURL = pathToFileURL(directory).href + '/'
const cache = captureEsmResolveCache(native, directoryURL)
/* v8 ignore else -- asynchronous loader hooks own a separate cache; loader-thread invalidation is deferred */
if (cache !== undefined) Map.prototype.clear.call(cache)
}
/** Restore every replaced Node method this instance installed. */
dispose(): void {
for (const restore of this.restorers.splice(0).reverse()) restore()
this.native = undefined
this.realDirectories.clear()
}
private isPathWithinDirectory(path: string, directory: string): boolean {
return path === directory || path.startsWith(join(directory, sep))
}
private isPathInInvalidatedDirectory(path: string): boolean {
const parent = dirname(path)
let real = this.realDirectories.get(parent)
if (real === undefined) {
try { real = realpathSync(parent) } catch (_error) {
// A missing directory keeps its configured spelling; it has no manifest to read through a link.
real = parent
}
this.realDirectories.set(parent, real)
}
const target = join(real, basename(path))
for (const directory of this.directories) if (this.isPathWithinDirectory(target, directory)) return true
return false
}
private isUrlInInvalidatedDirectory(url: string): boolean {
try { return this.isPathInInvalidatedDirectory(fileURLToPath(url)) } catch (_error) {
// Only file URLs can name an invalidated package directory.
return false
}
}
private installPackageHooks(): NativeAccess {
if (this.native !== undefined) return this.native
const native = loadNodeInternals()
this.native = native
const binding = createReplaceHelper(native.binding)
const reader = createReplaceHelper(native.reader)
const commonJs = createReplaceHelper<Pick<CommonJsModule, '_load'>>(native.cjs)
this.rawBinding = binding.rawImpl
this.rawReader = reader.rawImpl
this.rawCommonJs = commonJs.rawImpl
this.restorers.push(
binding.replaceMethod('readPackageJSON', this.hookBindingReadPackageJSON.bind(this)),
binding.replaceMethod('getPackageScopeConfig', this.hookBindingGetPackageScopeConfig.bind(this)),
binding.replaceMethod('getPackageType', this.hookBindingGetPackageType.bind(this)),
reader.replaceMethod('getNearestParentPackageJSON', this.hookReaderGetNearestParentPackageJSON.bind(this)),
commonJs.replaceMethod('_load', this.hookCommonJsLoad.bind(this)),
)
return native
}
private hookCommonJsLoad(...[request, parent, isMain, ...options]: Parameters<CommonJsModule['_load']>) {
const native = this.installPackageHooks()
// Absolute filenames bypass Node's private request cache without evicting evaluated modules.
const filename = request.startsWith('node:') || isBuiltin(request)
? request : native.cjs._resolveFilename(request, parent, isMain)
return this.rawCommonJs._load.call(native.cjs, filename, parent, isMain, ...options)
}
private hookBindingReadPackageJSON(path: string, isEsm?: boolean, base?: string, specifier?: string) {
return this.isPathInInvalidatedDirectory(path)
? this.readCachedPackageConfig(
path,
isEsm && base !== undefined ? { base, specifier } : undefined,
)?.serialized
: this.rawBinding.readPackageJSON(path, isEsm, base, specifier)
}
private hookBindingGetPackageScopeConfig(url: string) {
if (!this.isUrlInInvalidatedDirectory(url)) return this.rawBinding.getPackageScopeConfig(url)
const found = this.findNearestPackageConfig(fileURLToPath(url), true)
if (typeof found === 'string') return this.rawBinding.getPackageScopeConfig(pathToFileURL(found).href)
return found.configuration?.serialized ?? found.path
}
private hookBindingGetPackageType(url: string) {
if (!this.isUrlInInvalidatedDirectory(url)) return this.rawBinding.getPackageType(url)
const found = this.findNearestPackageConfig(fileURLToPath(url), true)
return typeof found === 'string'
? this.rawBinding.getPackageType(pathToFileURL(found).href)
: found.configuration?.data.type
}
private hookReaderGetNearestParentPackageJSON(path: string) {
if (!this.isPathInInvalidatedDirectory(path)) return this.rawReader.getNearestParentPackageJSON(path)
const found = this.findNearestPackageConfig(path, false)
if (typeof found === 'string') return this.rawReader.getNearestParentPackageJSON(found)
// A lookup that stops without a manifest keeps the running Node version's own absent result.
if (found.configuration === undefined) return this.rawReader.getNearestParentPackageJSON(path)
return { data: found.configuration.data, exists: true, path: found.path }
}
private readCachedPackageConfig(path: string, context?: ImportContext): Configuration | undefined {
if (!this.configurations.has(path)) this.configurations.set(path, this.readPackageConfigFromDisk(path, context))
return this.configurations.get(path)
}
/** A string names the first manifest outside the invalidated directories; Node answers for it. */
private findNearestPackageConfig(path: string, includeRoot: boolean): { path: string; configuration?: Configuration } | string {
let directory = dirname(path)
while (true) {
const manifest = join(directory, 'package.json')
const parent = dirname(directory)
if (basename(directory) === 'node_modules' || (!includeRoot && parent === directory)) return { path: manifest }
if (!this.isPathInInvalidatedDirectory(manifest)) return manifest
const configuration = this.readCachedPackageConfig(manifest)
if (configuration !== undefined) return { path: toNamespacedPath(manifest), configuration }
/* v8 ignore next -- root-scope queries run in an isolated child to contain non-termination */
if (parent === directory) return { path: manifest }
directory = parent
}
}
/** @param context - optional ESM import details used only when reporting invalid package configuration. */
private readPackageConfigFromDisk(path: string, context?: ImportContext): Configuration | undefined {
let bytes: Buffer
try { bytes = readFileSync(path) } catch (_error) {
// Node reports a missing manifest as absent configuration.
return undefined
}
const throwInvalidPackageConfig = (): never => {
// The native reader throws a plain Error carrying only this code.
const base = context?.base
const importing = base === undefined ? ''
: ` while importing "${context?.specifier}" from ${base.startsWith('file:') ? fileURLToPath(base) : base}`
throw Object.assign(new Error(`Invalid package config ${toNamespacedPath(path)}${importing}.`), { code: 'ERR_INVALID_PACKAGE_CONFIG' })
}
if (!isUtf8(bytes)) return throwInvalidPackageConfig()
const source = bytes.toString('utf8')
let parsed: unknown
try { parsed = JSON.parse(source.charCodeAt(0) === 0xfeff ? source.slice(1) : source) } catch (_error) {
return throwInvalidPackageConfig()
}
if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) return throwInvalidPackageConfig()
const fields = parsed as Record<string, unknown>
if (fields.name !== undefined && typeof fields.name !== 'string') return throwInvalidPackageConfig()
if (fields.type !== undefined && typeof fields.type !== 'string') return throwInvalidPackageConfig()
for (const key of ['name', 'type', 'exports', 'imports']) {
const value = fields[key]
if (typeof value === 'string' && !value.isWellFormed()) return throwInvalidPackageConfig()
}
const normalizePackageJsonField = (value: unknown): unknown =>
value !== null && (typeof value === 'string' || typeof value === 'object') ? value : undefined
const data: PackageConfig = {
...typeof fields.name === 'string' ? { name: fields.name } : {},
...typeof fields.main === 'string' && fields.main.isWellFormed() ? { main: fields.main } : {},
type: fields.type === 'module' || fields.type === 'commonjs' ? fields.type : 'none',
exports: normalizePackageJsonField(fields.exports),
imports: normalizePackageJsonField(fields.imports),
}
const serializePackageJsonField = (value: unknown): string | undefined =>
value === undefined ? undefined : typeof value === 'string' ? value : JSON.stringify(value)
return {
data,
serialized: [
data.name,
data.main,
data.type,
serializePackageJsonField(data.imports),
serializePackageJsonField(data.exports),
toNamespacedPath(path),
],
}
}
}
+22
View File
@@ -0,0 +1,22 @@
/** Isolate root-scope traversal; the filesystem root is read but never modified. */
import assert from 'node:assert/strict'
import { createRequire } from 'node:module'
import { join, parse } from 'node:path'
import { pathToFileURL } from 'node:url'
import { PackageManifests } from '../../src/package-manifest.ts'
const [root] = process.argv.slice(2)
const reader = createRequire(import.meta.url)('internal/modules/package_json_reader')
const source = join(root, 'missing', 'entry.js')
const url = pathToFileURL(source).href
const expectedScope = reader.getPackageScopeConfig(url)
const expectedNearest = reader.getNearestParentPackageJSON(source)
const manifests = new PackageManifests()
try {
await manifests.invalidate(join(parse(root).root, 'package.json'))
assert.deepEqual(reader.getPackageScopeConfig(url), expectedScope)
assert.deepEqual(reader.getNearestParentPackageJSON(source), expectedNearest)
process.stdout.write('scope traversal completed\n')
} finally {
manifests.dispose()
}
+107 -3
View File
@@ -58,6 +58,42 @@ async function fixture(version: 'v1' | 'v2' = 'v2') {
return { ctx, cache, imports, imported, reload, module, url, resolve }
}
it('does not reactivate a runtime removed during dependency traversal', async () => {
const { ctx, module, imports, imported, reload } = await fixture()
const dependency = module('dependency.mjs')
const original = { apply() {} }
const replacement = { apply: vi.fn() }
const job = module('plugin.mjs', original, [dependency])
const entry = ctx.loader.resolve(await ctx.loader.create({ name: job.url }))
await ctx.loader.await()
const fiber = entry.fiber!
const reached = Promise.withResolvers<undefined>()
const linked = Promise.withResolvers<ModuleJob[]>()
Object.defineProperty(job, 'linked', { configurable: true, get() {
reached.resolve(undefined)
return linked.promise
} })
onTestFinished(() => { linked.resolve([dependency]) })
imports.set(job.url, replacement)
imported.mockClear()
reload.stashed.add(dependency.url)
const operation = reload.partialReload()
try {
await reached.promise
ctx.registry.delete(original)
await fiber.await()
expect(ctx.registry.get(original)).toBeUndefined()
linked.resolve([dependency])
await operation
expect(imported).toHaveBeenCalledWith(job.url, expect.any(Function))
expect(replacement.apply).not.toHaveBeenCalled()
expect(ctx.registry.get(replacement)).toBeUndefined()
} finally {
linked.resolve([dependency])
await operation
}
})
it.each(['v1', 'v2'] as const)('replaces a %s module and retains the latest Loader entry configuration', async (version) => {
const { ctx, module, imports, reload } = await fixture(version)
const mounted: string[] = []
@@ -166,6 +202,34 @@ it('does not replace a plugin whose dependencies have not changed', async () =>
expect(imported).not.toHaveBeenCalled()
})
it('keeps a node_modules entry cached when it shares a runtime with a replaced source entry', async () => {
const { ctx, module, imports, imported, cache, reload } = await fixture()
const original = { apply: vi.fn() }
const replacement = { apply: vi.fn() }
const source = module('source.mjs', original)
const installedNamespace = { apply: original.apply }
const installed = module('node_modules/addon/index.mjs', installedNamespace)
await ctx.loader.root.update([
{ id: 'source', name: source.url },
{ id: 'installed', name: installed.url },
])
await ctx.loader.await()
expect(ctx.loader.resolve('source').fiber?.runtime).toBe(ctx.loader.resolve('installed').fiber?.runtime)
imports.set(source.url, replacement)
imported.mockClear()
reload.stashed.add(source.url)
await reload.partialReload()
expect(imported).toHaveBeenCalledExactlyOnceWith(source.url, expect.any(Function))
expect(replacement.apply).toHaveBeenCalledOnce()
expect(cache.get(installed.url)).toBe(installed)
expect(ctx.loader.resolve('installed').moduleNamespace).toBe(installedNamespace)
expect(ctx.loader.resolve('installed').fiber?.runtime?.callback).toBe(original.apply)
expect(ctx.loader.resolve('source').fiber?.runtime?.callback).toBe(replacement.apply)
expect(reload.accepted).not.toContain(installed.url)
})
it('replaces a plugin when a linked dependency changes, excluding framework modules', async () => {
const { ctx, module, imports, reload } = await fixture()
const dependency = module('dependency.mjs')
@@ -237,14 +301,15 @@ it('leaves disposed child instances to their replacing parent', async () => {
expect(entry.fiber?._config).toEqual({ value: 'entry' })
})
it('invalidates a disabled module without activating it', async () => {
const { ctx, module, imports, reload } = await fixture()
it.each(['v1', 'v2'] as const)('invalidates a disabled %s module without activating it', async (version) => {
const { ctx, module, imports, resolve, reload } = await fixture(version)
const job = module('disabled.mjs', { apply() {} })
const id = await ctx.loader.create({ name: job.url, disabled: true })
const apply = vi.fn()
imports.set(job.url, { apply })
reload.stashed.add(job.url)
await reload.partialReload()
expect(resolve.mock.calls).toEqual(version === 'v1' ? [[job.url, ctx.baseUrl, {}]] : [[job.url]])
expect(ctx.loader.resolve(id).fiber).toBeUndefined()
expect(apply).not.toHaveBeenCalled()
})
@@ -263,6 +328,45 @@ it('does not disturb an unchanged runtime after an earlier replacement fails', a
expect(untouched).toHaveBeenCalledOnce()
})
it('cleans up a shared replacement runtime once when a later plugin activation fails', async () => {
const { ctx, module, imports, cache, reload } = await fixture()
const mounted: string[] = []
const disposed: string[] = []
const original = { apply(_ctx: Context, config: { label: string }) { mounted.push(config.label) } }
const replacement = { apply(ctx: Context, config: { label: string }) {
ctx.effect(() => () => { disposed.push(config.label) })
} }
const first = module('shared.mjs', original)
const second = module('failing.mjs', { apply() {} })
await ctx.loader.root.update([
{ id: 'one', name: first.url, config: { label: 'one' } },
{ id: 'two', name: first.url, config: { label: 'two' } },
{ id: 'failing', name: second.url },
])
await ctx.loader.await()
imports.set(first.url, replacement)
imports.set(second.url, { apply() { throw new Error('later activation failed') } })
const remove = vi.spyOn(ctx.registry, 'delete')
const event = vi.fn()
ctx.on('hmr/reload', event)
reload.stashed.add(first.url)
reload.stashed.add(second.url)
await expect(reload.partialReload()).rejects.toThrow('later activation failed')
expect(disposed.sort()).toEqual(['one', 'two'])
expect(mounted).toEqual(['one', 'two', 'one', 'two'])
expect(remove.mock.calls.filter(([plugin]) => plugin === replacement.apply)).toHaveLength(1)
expect(ctx.registry.get(replacement)).toBeUndefined()
const restored = ctx.registry.get(original)
expect([...restored!.fibers]).toHaveLength(2)
expect(ctx.loader.resolve('one').fiber?.runtime).toBe(restored)
expect(ctx.loader.resolve('two').fiber?.runtime).toBe(restored)
expect(cache.get(first.url)).toBe(first)
expect(cache.get(second.url)).toBe(second)
expect(event).not.toHaveBeenCalled()
})
it('restores the prior plugin when the replacement does not export a plugin', async () => {
const { ctx, module, imported, reload } = await fixture()
const original = { apply: vi.fn() }
@@ -292,7 +396,7 @@ it('ignores framework entries and reports unresolved entry modules', async () =>
const framework = module('framework.mjs', { apply() {} })
const uncached = module('uncached.mjs', { apply() {} })
await ctx.loader.create({ name: framework.url })
await ctx.loader.create({ name: uncached.url })
await ctx.loader.create({ name: uncached.url, disabled: true })
await ctx.loader.await()
reload.externals.add(framework.url)
resolve.mockImplementation((name) => {
@@ -0,0 +1,669 @@
/** HMR dispatch of package manifest changes through a real Loader and a controlled watcher. */
import { randomUUID } from 'node:crypto'
import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { pathToFileURL } from 'node:url'
import { Context, type Plugin } from '@deepseek-ai/cordis'
import Loader from '@deepseek-ai/cordis-plugin-loader'
import Timer from '@deepseek-ai/cordis-plugin-timer'
import { FSWatcher, type ChokidarOptions } from 'chokidar'
import { expect, it, onTestFinished, vi } from 'vitest'
import Hmr from '../src/index.ts'
const watchers = vi.hoisted(() => [] as FSWatcher[])
// File notifications are delivered only when a test emits them.
vi.mock('chokidar', async (original) => {
const actual = await original<typeof import('chokidar')>()
return { ...actual, watch: (_paths: string | string[], options: ChokidarOptions = {}) => {
const watcher = new actual.FSWatcher(options)
watchers.push(watcher)
queueMicrotask(() => watcher.emit('ready'))
return watcher
} }
})
function file(path: string, source: string): void {
mkdirSync(dirname(path), { recursive: true })
writeFileSync(path, source)
}
function plugin(marker: string): string {
return `import { value } from '#dep';
export function apply(ctx) {
ctx.get('manifestTrace').push('start:${marker}:' + value);
ctx.effect(() => () => { ctx.get('manifestTrace').push('stop:${marker}'); });
}
`
}
function manifest(entry: string, dep: string): string {
return JSON.stringify({ name: 'addon', type: 'module', exports: `./${entry}.mjs`, imports: { '#dep': `./${dep}.mjs` } })
}
async function fixture(source = plugin('a'), options: { lateHmr?: boolean; packageJson?: string } = {}) {
const root = realpathSync.native(mkdtempSync(join(tmpdir(), 'dsh-hmr-manifest-dispatch-')))
const ctx = new Context()
onTestFinished(async () => {
try { await ctx.fiber.dispose() } finally { rmSync(root, { recursive: true, force: true }) }
})
const pkg = join(root, 'plugins', 'addon')
file(join(pkg, 'package.json'), options.packageJson ?? manifest('a', 'dep-1'))
file(join(pkg, 'a.mjs'), source)
file(join(pkg, 'b.mjs'), plugin('b'))
file(join(pkg, 'dep-1.mjs'), 'export const value = 1')
file(join(pkg, 'dep-2.mjs'), 'export const value = 2')
file(join(root, 'app', 'package.json'), '{"type":"module"}')
mkdirSync(join(root, 'app', 'node_modules'), { recursive: true })
symlinkSync(pkg, join(root, 'app', 'node_modules', 'addon'), process.platform === 'win32' ? 'junction' : 'dir')
const trace: string[] = []
ctx.baseUrl = pathToFileURL(join(root, 'app')).href + '/'
ctx.provide('manifestTrace', trace)
await ctx.plugin(Loader)
await ctx.plugin(Timer)
const startHmr = async () => {
const start = watchers.length
const fiber = (await ctx.plugin(Hmr, { root: ['../plugins'], ignored: [], debounce: 0 })).ctx.fiber
return { fiber, watcher: watchers.slice(start).at(-1)! }
}
const initialHmr = options.lateHmr ? undefined : await startHmr()
const entry = ctx.loader.resolve(await ctx.loader.create({ name: 'addon' }))
await ctx.loader.await()
let hmr = initialHmr ?? await startHmr()
const warn = vi.spyOn(ctx.logger, 'warn')
onTestFinished(() => { warn.mockRestore() })
const restartHmr = async () => {
await hmr.fiber.dispose()
hmr = await startHmr()
}
const emit = async (...paths: string[]) => {
const dispatch = vi.spyOn(ctx.hmr, 'runExclusive')
try {
for (const path of paths) hmr.watcher.emit('change', path)
await vi.waitFor(() => { expect(dispatch).toHaveBeenCalledOnce() })
const result = dispatch.mock.results[0]!
expect(result.type).toBe('return')
await result.value
} finally {
dispatch.mockRestore()
}
}
return { ctx, pkg, trace, emit, entry, warn, restartHmr }
}
it('does not reload for a manifest alone and resolves the new entry when the Loader entry restarts', async () => {
const f = await fixture()
expect(f.trace).toEqual(['start:a:1'])
file(join(f.pkg, 'package.json'), manifest('b', 'dep-2'))
await f.emit(join(f.pkg, 'package.json'))
expect(f.trace).toEqual(['start:a:1'])
await f.entry.update({ disabled: true })
await f.entry.update({ disabled: false })
await f.ctx.loader.await()
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:b:2'])
})
const jsonPlugin = `import manifest from './package.json' with { type: 'json' };
export function apply(ctx) {
ctx.get('manifestTrace').push('start:json:' + manifest.imports['#dep']);
ctx.effect(() => () => { ctx.get('manifestTrace').push('stop:json'); });
}
`
it('reloads a plugin that imports its changed manifest as a JSON module', async () => {
const f = await fixture(jsonPlugin)
expect(f.trace).toEqual(['start:json:./dep-1.mjs'])
file(join(f.pkg, 'package.json'), manifest('a', 'dep-2'))
await f.emit(join(f.pkg, 'package.json'))
expect(f.trace).toEqual(['start:json:./dep-1.mjs', 'stop:json', 'start:json:./dep-2.mjs'])
})
it('requests a host reload for a changed manifest in the host module graph', async () => {
const f = await fixture(jsonPlugin)
const filename = join(f.pkg, 'package.json')
const externals = Reflect.get(f.ctx.hmr, 'externals') as Set<string>
externals.add(pathToFileURL(filename).href)
const exit = vi.spyOn(f.ctx.loader, 'exit').mockImplementation(() => {})
onTestFinished(() => { exit.mockRestore() })
file(filename, manifest('a', 'dep-2'))
await f.emit(filename)
expect(exit).toHaveBeenCalledOnce()
expect(f.trace).toEqual(['start:json:./dep-1.mjs'])
})
it('leaves a configuration-owned manifest to its dedicated watcher', async () => {
const f = await fixture(jsonPlugin)
const filename = join(f.pkg, 'package.json')
const dispose = await f.ctx.hmr.watchConfig(filename, async () => {})
try {
file(filename, manifest('a', 'dep-2'))
await f.emit(filename)
expect(f.trace).toEqual(['start:json:./dep-1.mjs'])
} finally {
await dispose()
}
})
it('resolves package imports from the changed manifest when a source change reloads the plugin', async () => {
const f = await fixture()
file(join(f.pkg, 'package.json'), manifest('a', 'dep-2'))
file(join(f.pkg, 'a.mjs'), `${plugin('a')}// edited\n`)
await f.emit(join(f.pkg, 'a.mjs'), join(f.pkg, 'package.json'))
await f.ctx.loader.await()
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a:2'])
})
it('keeps cached package imports when only source changes', async () => {
const f = await fixture()
file(join(f.pkg, 'package.json'), manifest('a', 'dep-2'))
file(join(f.pkg, 'a.mjs'), `${plugin('a')}// edited\n`)
await f.emit(join(f.pkg, 'a.mjs'))
await f.ctx.loader.await()
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a:1'])
})
it('leaves manifests below node_modules to their package lifecycle', async () => {
const f = await fixture()
const installed = join(dirname(f.pkg), 'node_modules', 'dep', 'package.json')
file(installed, '{"name":"dep"}')
const invalidate = vi.spyOn(Reflect.get(f.ctx.hmr, 'manifests') as { invalidate(path: string): void }, 'invalidate')
onTestFinished(() => { invalidate.mockRestore() })
await f.emit(installed)
expect(invalidate).not.toHaveBeenCalled()
})
it.each(['exports', 'main'] as const)('keeps reloading the loaded entry after %s moves and selects the new entry on restart', async (field) => {
const packageJson = (entry: string, dep: string) => JSON.stringify({
name: 'addon', type: 'module', [field]: `./${entry}.mjs`, imports: { '#dep': `./${dep}.mjs` },
})
const f = await fixture(plugin('a'), { packageJson: packageJson('a', 'dep-1') })
const originalNamespace = f.entry.moduleNamespace
file(join(f.pkg, 'package.json'), packageJson('b', 'dep-2'))
await f.emit(join(f.pkg, 'package.json'))
expect(f.trace).toEqual(['start:a:1'])
expect(f.entry.moduleNamespace).toBe(originalNamespace)
file(join(f.pkg, 'a.mjs'), plugin('a-edited'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a-edited:2'])
expect(f.entry.moduleNamespace).not.toBe(originalNamespace)
expect(f.entry.moduleNamespace).toBe(await f.ctx.loader.import(pathToFileURL(join(f.pkg, 'a.mjs')).href))
await f.entry.update({ disabled: true })
await f.entry.update({ disabled: false })
await f.ctx.loader.await()
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a-edited:2', 'stop:a-edited', 'start:b:2'])
expect(f.warn).not.toHaveBeenCalled()
})
it('retains the loaded entry through repeated reloads when the new entry is already cached', async () => {
const f = await fixture()
file(join(f.pkg, 'package.json'), manifest('b', 'dep-2'))
await f.emit(join(f.pkg, 'package.json'))
await f.ctx.loader.import('addon')
const originalNamespace = f.entry.moduleNamespace
file(join(f.pkg, 'a.mjs'), plugin('a-first'))
await f.emit(join(f.pkg, 'a.mjs'))
const firstNamespace = f.entry.moduleNamespace
expect(firstNamespace).not.toBe(originalNamespace)
file(join(f.pkg, 'a.mjs'), plugin('a-second'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(f.entry.moduleNamespace).not.toBe(firstNamespace)
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a-first:2', 'stop:a-first', 'start:a-second:2'])
expect(f.warn).not.toHaveBeenCalled()
})
it.each(['apply', 'apply as default'])('retains a re-export entry using its namespace when it exports %s', async (exported) => {
const f = await fixture(`export { ${exported} } from './b.mjs';\n`)
const implementation: unknown = await f.ctx.loader.import(pathToFileURL(join(f.pkg, 'b.mjs')).href)
expect(implementation).toMatchObject({ apply: f.entry.fiber?.runtime?.callback })
expect(f.entry.moduleNamespace).not.toBe(implementation)
file(join(f.pkg, 'c.mjs'), plugin('c'))
file(join(f.pkg, 'package.json'), manifest('c', 'dep-1'))
await f.emit(join(f.pkg, 'package.json'))
file(join(f.pkg, 'dep-1.mjs'), 'export const value = 2')
await f.emit(join(f.pkg, 'dep-1.mjs'))
expect(f.trace).toEqual(['start:b:1', 'stop:b', 'start:b:2'])
expect(f.entry.moduleNamespace).toBe(await f.ctx.loader.import(pathToFileURL(join(f.pkg, 'a.mjs')).href))
await f.entry.update({ disabled: true })
await f.entry.update({ disabled: false })
await f.ctx.loader.await()
expect(f.trace).toEqual(['start:b:1', 'stop:b', 'start:b:2', 'stop:b', 'start:c:2'])
expect(f.warn).not.toHaveBeenCalled()
})
it('tracks an entry loaded before HMR starts', async () => {
const f = await fixture(plugin('a'), { lateHmr: true })
file(join(f.pkg, 'package.json'), manifest('b', 'dep-1'))
await f.emit(join(f.pkg, 'package.json'))
file(join(f.pkg, 'a.mjs'), plugin('a-late'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a-late:1'])
expect(f.warn).not.toHaveBeenCalled()
})
it('keeps entry identity when HMR is recreated between source reloads', async () => {
const f = await fixture()
file(join(f.pkg, 'a.mjs'), plugin('a-first'))
await f.emit(join(f.pkg, 'a.mjs'))
const firstNamespace = f.entry.moduleNamespace
await f.restartHmr()
expect(f.entry.moduleNamespace).toBe(firstNamespace)
file(join(f.pkg, 'a.mjs'), plugin('a-second'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(f.entry.moduleNamespace).not.toBe(firstNamespace)
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a-first:1', 'stop:a-first', 'start:a-second:1'])
expect(f.warn).not.toHaveBeenCalled()
})
it('leaves an unimported disabled entry without a Node cache entry alone', async () => {
const f = await fixture()
const disabled = f.ctx.loader.resolve(await f.ctx.loader.create({
name: pathToFileURL(join(f.pkg, 'b.mjs')).href, disabled: true,
}))
expect(disabled.moduleNamespace).toBeUndefined()
file(join(f.pkg, 'a.mjs'), plugin('a-edited'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(disabled.moduleNamespace).toBeUndefined()
expect(disabled.fiber).toBeUndefined()
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a-edited:1'])
expect(f.warn).not.toHaveBeenCalled()
})
it('leaves a cordis builtin outside the Node namespace index without reporting an error', async () => {
const f = await fixture()
const builtin = { apply: vi.fn() }
f.ctx.loader.builtins.fixture = builtin
const entry = f.ctx.loader.resolve(await f.ctx.loader.create({ name: 'cordis:fixture' }))
await f.ctx.loader.await()
expect(entry.moduleNamespace).toBe(builtin)
file(join(f.pkg, 'a.mjs'), plugin('a-edited'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(builtin.apply).toHaveBeenCalledOnce()
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a-edited:1'])
expect(f.warn).not.toHaveBeenCalled()
})
it('updates disabled entry namespaces through repeated reloads without activating them', async () => {
const f = await fixture()
const originalNamespace = f.entry.moduleNamespace
await f.entry.update({ disabled: true })
await f.ctx.loader.await()
file(join(f.pkg, 'a.mjs'), plugin('disabled-first'))
await f.emit(join(f.pkg, 'a.mjs'))
const firstNamespace = f.entry.moduleNamespace
expect(firstNamespace).not.toBe(originalNamespace)
expect(firstNamespace).toBe(await f.ctx.loader.import('addon'))
expect(f.trace).toEqual(['start:a:1', 'stop:a'])
file(join(f.pkg, 'a.mjs'), plugin('disabled-second'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(f.entry.moduleNamespace).not.toBe(firstNamespace)
expect(f.trace).toEqual(['start:a:1', 'stop:a'])
expect(f.warn).not.toHaveBeenCalled()
await f.entry.update({ disabled: false })
await f.ctx.loader.await()
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:disabled-second:1'])
})
it('reports a resolution error for an unimported entry through the existing warning path', async () => {
const f = await fixture()
await f.ctx.loader.create({ name: './missing-plugin.mjs', disabled: true })
file(join(f.pkg, 'a.mjs'), plugin('a-edited'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(f.warn).toHaveBeenCalledWith(expect.objectContaining({ code: 'ERR_MODULE_NOT_FOUND' }))
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a-edited:1'])
})
it('reports a recorded namespace missing from the cache while reloading another entry', async () => {
const f = await fixture()
const originalFiber = f.entry.fiber
const originalNamespace = f.entry.moduleNamespace
await f.ctx.loader.create({ name: pathToFileURL(join(f.pkg, 'b.mjs')).href })
await f.ctx.loader.await()
expect(Map.prototype.delete.call(f.ctx.loader.internal!.loadCache, pathToFileURL(join(f.pkg, 'a.mjs')).href)).toBe(true)
file(join(f.pkg, 'b.mjs'), plugin('b-edited'))
await f.emit(join(f.pkg, 'b.mjs'))
expect(f.warn).toHaveBeenCalledWith(expect.objectContaining({
message: `HMR cannot locate the loaded module for addon from ${f.ctx.baseUrl}`,
}))
expect(f.entry.fiber).toBe(originalFiber)
expect(f.entry.moduleNamespace).toBe(originalNamespace)
expect(f.trace).toEqual(['start:a:1', 'start:b:1', 'stop:b', 'start:b-edited:1'])
})
it.each(['import', 'activation'] as const)('retains the prior namespace after replacement %s fails and can reload again', async (phase) => {
const f = await fixture()
const originalNamespace = f.entry.moduleNamespace
const originalCallback = f.entry.fiber?.runtime?.callback
const message = `replacement ${phase} failed`
file(join(f.pkg, 'a.mjs'), phase === 'import'
? `throw new Error(${JSON.stringify(message)});\n${plugin('failed')}`
: `export function apply() { throw new Error(${JSON.stringify(message)}); }`)
await expect(f.emit(join(f.pkg, 'a.mjs'))).rejects.toThrow(message)
expect(f.entry.moduleNamespace).toBe(originalNamespace)
expect(await f.ctx.loader.import('addon')).toBe(originalNamespace)
expect(f.entry.fiber?.runtime?.callback).toBe(originalCallback)
expect(f.trace).toEqual(phase === 'import' ? ['start:a:1'] : ['start:a:1', 'stop:a', 'start:a:1'])
file(join(f.pkg, 'a.mjs'), plugin('recovered'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(f.entry.moduleNamespace).not.toBe(originalNamespace)
expect(f.trace.at(-1)).toBe('start:recovered:1')
})
it('reloads a native CommonJS entry and retains its new import namespace', async () => {
const f = await fixture()
const filename = join(f.pkg, 'common.cjs')
const source = (version: number) => `exports.apply = ctx => {
ctx.get('manifestTrace').push('start:cjs:${version}');
ctx.effect(() => () => { ctx.get('manifestTrace').push('stop:cjs:${version}'); });
};`
file(filename, source(1))
const entry = f.ctx.loader.resolve(await f.ctx.loader.create({ name: pathToFileURL(filename).href }))
await f.ctx.loader.await()
const originalNamespace = entry.moduleNamespace
file(filename, source(2))
await f.emit(filename)
expect(entry.moduleNamespace).not.toBe(originalNamespace)
expect(entry.moduleNamespace).toBe(await f.ctx.loader.import(pathToFileURL(filename).href))
expect(f.trace).toEqual(['start:a:1', 'start:cjs:1', 'stop:cjs:1', 'start:cjs:2'])
expect(f.warn).not.toHaveBeenCalled()
})
it('handles an entry still evaluating while another entry reloads', async () => {
const f = await fixture()
const entered = Promise.withResolvers<undefined>()
const release = Promise.withResolvers<undefined>()
const key = 'dsh-hmr-pending-entry-' + randomUUID()
Reflect.set(globalThis, key, { entered: () => { entered.resolve(undefined) }, wait: release.promise })
f.ctx.provide('releasePendingEntry', () => { release.resolve(undefined) })
const filename = join(f.pkg, 'pending.mjs')
file(filename, `const gate = globalThis[${JSON.stringify(key)}]; gate.entered(); await gate.wait;\n${plugin('pending')}`)
const loading = f.ctx.loader.create({ name: pathToFileURL(filename).href })
onTestFinished(async () => {
release.resolve(undefined)
try { await loading } finally { Reflect.deleteProperty(globalThis, key) }
})
try {
await entered.promise
file(join(f.pkg, 'a.mjs'), plugin('a-edited').replace('export function apply(ctx) {',
"export function apply(ctx) { ctx.get('releasePendingEntry')();"))
await f.emit(join(f.pkg, 'a.mjs'))
const entry = f.ctx.loader.resolve(await loading)
await f.ctx.loader.await()
expect(entry.moduleNamespace).toBe(await f.ctx.loader.import(pathToFileURL(filename).href))
expect(f.trace).toEqual(['start:a:1', 'stop:a', 'start:a-edited:1', 'start:pending:1'])
expect(f.warn).not.toHaveBeenCalled()
} finally {
release.resolve(undefined)
await loading
}
})
it('keeps an active entry namespace when a later same-name entry has never been enabled', async () => {
const f = await fixture()
await f.ctx.loader.root.update([
...f.ctx.loader.root.data,
{ id: 'disabled-shadow', name: 'addon', disabled: true },
])
await f.ctx.loader.await()
const disabled = f.ctx.loader.resolve('disabled-shadow')
expect(disabled.moduleNamespace).toBeUndefined()
expect(disabled.fiber).toBeUndefined()
const before = [...f.trace]
file(join(f.pkg, 'package.json'), manifest('b', 'dep-2'))
await f.emit(join(f.pkg, 'package.json'))
expect(f.trace).toEqual(before)
file(join(f.pkg, 'a.mjs'), plugin('a-edited'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(f.trace).toEqual([...before, 'stop:a', 'start:a-edited:2'])
expect(disabled.fiber).toBeUndefined()
expect(f.warn).not.toHaveBeenCalled()
})
type DispatchFixture = Awaited<ReturnType<typeof fixture>>
async function addConfiguredEntry(f: DispatchFixture, id: string, name: string, disabled = false) {
const options = { id, name, disabled }
const entry = f.ctx.loader.resolve(await f.ctx.loader.create(options))
await f.ctx.loader.await()
return entry
}
function cachedNamespace(f: DispatchFixture, filename: string): unknown {
const url = pathToFileURL(join(f.pkg, filename)).href
const job = f.ctx.loader.internal!.loadCache.get(url)
expect(job?.module).toBeDefined()
return job!.module!.getNamespace()
}
function liveFibers(runtime: Plugin.Runtime | null | undefined) {
expect(runtime).toBeDefined()
expect(runtime).not.toBeNull()
return [...runtime!.fibers].filter(fiber => fiber.uid !== null)
}
async function sharedEntryFixture(kind: 'default' | 'named') {
const source = kind === 'default'
? "export { apply as default } from './b.mjs';\n"
: "export { apply } from './b.mjs';\n"
const f = await fixture(source)
const secondFile = kind === 'default' ? 'c.mjs' : 'b.mjs'
if (kind === 'default') file(join(f.pkg, secondFile), source)
const second = await addConfiguredEntry(f, 'shared-second', pathToFileURL(join(f.pkg, secondFile)).href)
const firstPlugin = f.ctx.loader.unwrapExports(f.entry.moduleNamespace) as Plugin
const secondPlugin = f.ctx.loader.unwrapExports(second.moduleNamespace) as Plugin
expect(f.entry.moduleNamespace).not.toBe(second.moduleNamespace)
expect(f.entry.moduleNamespace).toBe(cachedNamespace(f, 'a.mjs'))
expect(second.moduleNamespace).toBe(cachedNamespace(f, secondFile))
if (kind === 'default') expect(firstPlugin).toBe(secondPlugin)
else expect(firstPlugin).not.toBe(secondPlugin)
expect(f.entry.fiber?.runtime).toBe(second.fiber?.runtime)
expect(liveFibers(f.entry.fiber?.runtime)).toHaveLength(2)
expect(f.trace).toEqual(['start:b:1', 'start:b:1'])
return { ...f, second, secondFile, firstPlugin }
}
it('reloads both active namespaces for one package name independently after exports move', async () => {
const f = await fixture()
const firstNamespace = f.entry.moduleNamespace
const firstRuntime = f.entry.fiber?.runtime
file(join(f.pkg, 'package.json'), manifest('b', 'dep-2'))
await f.emit(join(f.pkg, 'package.json'))
const second = await addConfiguredEntry(f, 'active-after-move', 'addon')
const secondNamespace = second.moduleNamespace
const secondRuntime = second.fiber?.runtime
expect(firstNamespace).not.toBe(secondNamespace)
expect(firstRuntime).not.toBe(secondRuntime)
expect(liveFibers(firstRuntime)).toHaveLength(1)
expect(liveFibers(secondRuntime)).toHaveLength(1)
expect(f.trace).toEqual(['start:a:1', 'start:b:2'])
file(join(f.pkg, 'a.mjs'), plugin('a-independent'))
await f.emit(join(f.pkg, 'a.mjs'))
const reloadedFirst = f.entry.moduleNamespace
expect(reloadedFirst).not.toBe(firstNamespace)
expect(reloadedFirst).toBe(cachedNamespace(f, 'a.mjs'))
expect(second.moduleNamespace).toBe(secondNamespace)
expect(second.fiber?.runtime).toBe(secondRuntime)
expect(f.trace).toEqual(['start:a:1', 'start:b:2', 'stop:a', 'start:a-independent:2'])
file(join(f.pkg, 'b.mjs'), plugin('b-independent'))
await f.emit(join(f.pkg, 'b.mjs'))
expect(f.entry.moduleNamespace).toBe(reloadedFirst)
expect(second.moduleNamespace).not.toBe(secondNamespace)
expect(second.moduleNamespace).toBe(cachedNamespace(f, 'b.mjs'))
expect(liveFibers(f.entry.fiber?.runtime)).toHaveLength(1)
expect(liveFibers(second.fiber?.runtime)).toHaveLength(1)
expect(f.trace).toEqual([
'start:a:1', 'start:b:2', 'stop:a', 'start:a-independent:2', 'stop:b', 'start:b-independent:2',
])
expect(f.warn).not.toHaveBeenCalled()
})
it('refreshes both default re-export namespaces sharing one plugin without multiplying instances', async () => {
const f = await sharedEntryFixture('default')
const firstNamespace = f.entry.moduleNamespace
const secondNamespace = f.second.moduleNamespace
const oldFibers = liveFibers(f.entry.fiber?.runtime)
const before = f.trace.length
file(join(f.pkg, 'dep-1.mjs'), 'export const value = 2')
await f.emit(join(f.pkg, 'dep-1.mjs'))
expect(f.entry.moduleNamespace).not.toBe(firstNamespace)
expect(f.second.moduleNamespace).not.toBe(secondNamespace)
expect(f.entry.moduleNamespace).not.toBe(f.second.moduleNamespace)
expect(f.entry.moduleNamespace).toBe(cachedNamespace(f, 'a.mjs'))
expect(f.second.moduleNamespace).toBe(cachedNamespace(f, 'c.mjs'))
expect(f.ctx.loader.unwrapExports(f.entry.moduleNamespace)).toBe(f.ctx.loader.unwrapExports(f.second.moduleNamespace))
expect(f.entry.fiber?.runtime).toBe(f.second.fiber?.runtime)
expect(liveFibers(f.entry.fiber?.runtime)).toHaveLength(2)
expect(oldFibers.every(fiber => fiber.uid === null)).toBe(true)
expect(f.trace.slice(before).sort()).toEqual(['start:b:2', 'start:b:2', 'stop:b', 'stop:b'])
expect(f.warn).not.toHaveBeenCalled()
})
it('refreshes named re-export and implementation namespaces sharing a callback without multiplying instances', async () => {
const f = await sharedEntryFixture('named')
const firstNamespace = f.entry.moduleNamespace
const secondNamespace = f.second.moduleNamespace
const oldFibers = liveFibers(f.entry.fiber?.runtime)
const before = f.trace.length
file(join(f.pkg, 'b.mjs'), plugin('implementation-updated'))
await f.emit(join(f.pkg, 'b.mjs'))
expect(f.entry.moduleNamespace).not.toBe(firstNamespace)
expect(f.second.moduleNamespace).not.toBe(secondNamespace)
expect(f.entry.moduleNamespace).toBe(cachedNamespace(f, 'a.mjs'))
expect(f.second.moduleNamespace).toBe(cachedNamespace(f, 'b.mjs'))
expect(f.ctx.loader.unwrapExports(f.entry.moduleNamespace)).not.toBe(f.ctx.loader.unwrapExports(f.second.moduleNamespace))
expect(f.entry.fiber?.runtime).toBe(f.second.fiber?.runtime)
expect(liveFibers(f.entry.fiber?.runtime)).toHaveLength(2)
expect(oldFibers.every(fiber => fiber.uid === null)).toBe(true)
expect(f.trace.slice(before).sort()).toEqual([
'start:implementation-updated:1', 'start:implementation-updated:1', 'stop:b', 'stop:b',
])
expect(f.warn).not.toHaveBeenCalled()
})
function activeTraceCount(trace: string[], marker: string): number {
return trace.filter(event => event.startsWith(`start:${marker}:`)).length
- trace.filter(event => event === `stop:${marker}`).length
}
it('assigns each entry its own replacement when a shared runtime splits into distinct callbacks', async () => {
const f = await sharedEntryFixture('default')
const firstNamespace = f.entry.moduleNamespace
const secondNamespace = f.second.moduleNamespace
const oldFibers = liveFibers(f.entry.fiber?.runtime)
const before = f.trace.length
file(join(f.pkg, 'a.mjs'), plugin('a-split'))
file(join(f.pkg, 'c.mjs'), plugin('c-split'))
await f.emit(join(f.pkg, 'a.mjs'), join(f.pkg, 'c.mjs'))
expect(f.entry.moduleNamespace).not.toBe(firstNamespace)
expect(f.second.moduleNamespace).not.toBe(secondNamespace)
expect(f.entry.moduleNamespace).toBe(cachedNamespace(f, 'a.mjs'))
expect(f.second.moduleNamespace).toBe(cachedNamespace(f, 'c.mjs'))
expect(f.entry.moduleNamespace).toMatchObject({ apply: f.entry.fiber?.runtime?.callback })
expect(f.second.moduleNamespace).toMatchObject({ apply: f.second.fiber?.runtime?.callback })
expect(f.entry.fiber?.runtime).not.toBe(f.second.fiber?.runtime)
expect(liveFibers(f.entry.fiber?.runtime)).toHaveLength(1)
expect(liveFibers(f.second.fiber?.runtime)).toHaveLength(1)
expect(oldFibers.every(fiber => fiber.uid === null)).toBe(true)
expect(f.trace.slice(before).sort()).toEqual(['start:a-split:1', 'start:c-split:1', 'stop:b', 'stop:b'])
expect(f.warn).not.toHaveBeenCalled()
})
it('restores shared entries and a programmatic instance when split replacements are ambiguous', async () => {
const f = await sharedEntryFixture('default')
const firstNamespace = f.entry.moduleNamespace
const secondNamespace = f.second.moduleNamespace
const programmatic = (await f.ctx.plugin(f.firstPlugin)).ctx.fiber
expect(programmatic.entry).toBeUndefined()
expect(liveFibers(f.ctx.registry.get(f.firstPlugin))).toHaveLength(3)
file(join(f.pkg, 'a.mjs'), plugin('a-split'))
file(join(f.pkg, 'c.mjs'), plugin('c-split'))
await expect(f.emit(join(f.pkg, 'a.mjs'), join(f.pkg, 'c.mjs'))).rejects.toThrow(
'HMR replacement is ambiguous for a plugin instance without a Loader entry',
)
expect(f.entry.moduleNamespace).toBe(firstNamespace)
expect(f.second.moduleNamespace).toBe(secondNamespace)
expect(cachedNamespace(f, 'a.mjs')).toBe(firstNamespace)
expect(cachedNamespace(f, 'c.mjs')).toBe(secondNamespace)
const restored = f.ctx.registry.get(f.firstPlugin)
const fibers = liveFibers(restored)
expect(fibers).toHaveLength(3)
expect(fibers.filter(fiber => fiber.entry === undefined)).toHaveLength(1)
expect(f.entry.fiber?.runtime).toBe(restored)
expect(f.second.fiber?.runtime).toBe(restored)
expect(activeTraceCount(f.trace, 'b')).toBe(3)
expect(activeTraceCount(f.trace, 'a-split')).toBe(0)
expect(activeTraceCount(f.trace, 'c-split')).toBe(0)
})
it.each(['import', 'activation'] as const)('restores every shared entry when one replacement %s fails and can reload again', async (phase) => {
const f = await sharedEntryFixture('default')
const firstNamespace = f.entry.moduleNamespace
const secondNamespace = f.second.moduleNamespace
const message = `shared replacement ${phase} failed`
file(join(f.pkg, 'a.mjs'), plugin('a-replacement'))
file(join(f.pkg, 'c.mjs'), phase === 'import'
? `throw new Error(${JSON.stringify(message)});\n${plugin('c-failed')}`
: `export function apply() { throw new Error(${JSON.stringify(message)}); }`)
await expect(f.emit(join(f.pkg, 'a.mjs'), join(f.pkg, 'c.mjs'))).rejects.toThrow(message)
expect(f.entry.moduleNamespace).toBe(firstNamespace)
expect(f.second.moduleNamespace).toBe(secondNamespace)
expect(cachedNamespace(f, 'a.mjs')).toBe(firstNamespace)
expect(cachedNamespace(f, 'c.mjs')).toBe(secondNamespace)
const restored = f.ctx.registry.get(f.firstPlugin)
expect(f.entry.fiber?.runtime).toBe(restored)
expect(f.second.fiber?.runtime).toBe(restored)
expect(liveFibers(restored)).toHaveLength(2)
expect(activeTraceCount(f.trace, 'b')).toBe(2)
expect(activeTraceCount(f.trace, 'a-replacement')).toBe(0)
expect(activeTraceCount(f.trace, 'c-failed')).toBe(0)
file(join(f.pkg, 'c.mjs'), plugin('c-recovered'))
await f.emit(join(f.pkg, 'a.mjs'), join(f.pkg, 'c.mjs'))
expect(f.entry.moduleNamespace).not.toBe(firstNamespace)
expect(f.second.moduleNamespace).not.toBe(secondNamespace)
expect(f.entry.moduleNamespace).toBe(cachedNamespace(f, 'a.mjs'))
expect(f.second.moduleNamespace).toBe(cachedNamespace(f, 'c.mjs'))
expect(f.entry.moduleNamespace).toMatchObject({ apply: f.entry.fiber?.runtime?.callback })
expect(f.second.moduleNamespace).toMatchObject({ apply: f.second.fiber?.runtime?.callback })
expect(f.entry.fiber?.runtime).not.toBe(f.second.fiber?.runtime)
expect(liveFibers(f.entry.fiber?.runtime)).toHaveLength(1)
expect(liveFibers(f.second.fiber?.runtime)).toHaveLength(1)
expect(activeTraceCount(f.trace, 'b')).toBe(0)
expect(activeTraceCount(f.trace, 'a-replacement')).toBe(1)
expect(activeTraceCount(f.trace, 'c-recovered')).toBe(1)
})
it('deduplicates a repeated namespace without activating a same-name uninitialized entry', async () => {
const f = await fixture()
const second = await addConfiguredEntry(f, 'duplicate-active', 'addon')
const disabled = await addConfiguredEntry(f, 'duplicate-disabled', 'addon', true)
const originalNamespace = f.entry.moduleNamespace
expect(second.moduleNamespace).toBe(originalNamespace)
expect(disabled.moduleNamespace).toBeUndefined()
expect(disabled.fiber).toBeUndefined()
expect(liveFibers(f.entry.fiber?.runtime)).toHaveLength(2)
file(join(f.pkg, 'package.json'), manifest('b', 'dep-2'))
await f.emit(join(f.pkg, 'package.json'))
const before = f.trace.length
file(join(f.pkg, 'a.mjs'), plugin('deduplicated'))
await f.emit(join(f.pkg, 'a.mjs'))
expect(f.entry.moduleNamespace).not.toBe(originalNamespace)
expect(f.entry.moduleNamespace).toBe(cachedNamespace(f, 'a.mjs'))
expect(second.moduleNamespace).toBe(f.entry.moduleNamespace)
expect(f.entry.fiber?.runtime).toBe(second.fiber?.runtime)
expect(liveFibers(f.entry.fiber?.runtime)).toHaveLength(2)
expect(disabled.moduleNamespace).toBeUndefined()
expect(disabled.fiber).toBeUndefined()
expect(f.trace.slice(before).sort()).toEqual([
'start:deduplicated:2', 'start:deduplicated:2', 'stop:a', 'stop:a',
])
expect(f.warn).not.toHaveBeenCalled()
})
@@ -0,0 +1,457 @@
/** Package configuration read by Node's resolver follows an invalidated manifest on disk. */
import { spawn } from 'node:child_process'
import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, unlinkSync, writeFileSync } from 'node:fs'
import NodeModule, { createRequire, registerHooks } from 'node:module'
import { tmpdir } from 'node:os'
import { dirname, join, toNamespacedPath } from 'node:path'
import { fileURLToPath, pathToFileURL } from 'node:url'
import { ModuleLoader } from '@deepseek-ai/cordis-plugin-loader'
import { afterEach, describe, expect, it } from 'vitest'
import { PackageManifests } from '../src/package-manifest.ts'
const require = createRequire(import.meta.url)
const addon = require('node-addon-require-builtin') as { requireBuiltin(id: string): unknown }
const reader = addon.requireBuiltin('internal/modules/package_json_reader') as {
read: (path: string, options?: { isESM: boolean; base: string; specifier: string }) => { exists: boolean; type: string }
getPackageScopeConfig: (url: string) => { exists: boolean; type: string; pjsonPath: string }
getNearestParentPackageJSON: (path: string) => { data: { type: string }; path: string } | undefined
}
const formats = addon.requireBuiltin('internal/modules/esm/get_format') as {
defaultGetFormatWithoutErrors(url: URL, context?: object): string | null
}
const binding = (addon.requireBuiltin('internal/bootstrap/realm') as {
internalBinding(id: 'modules'): Record<string, unknown>
}).internalBinding('modules')
const cleanup: Array<() => void | Promise<void>> = []
afterEach(async () => { for (const dispose of cleanup.splice(0).reverse()) await dispose() })
function file(path: string, source: string): void {
mkdirSync(dirname(path), { recursive: true })
writeFileSync(path, source)
}
function fixture() {
const root = realpathSync.native(mkdtempSync(join(tmpdir(), 'dsh-hmr-manifest-')))
cleanup.push(() => { rmSync(root, { recursive: true, force: true }) })
const dir = join(root, 'packages', 'pkg')
const manifest = join(dir, 'package.json')
const importer = join(root, 'importer', 'entry.mjs')
file(join(root, 'importer', 'package.json'), '{"type":"module","dependencies":{"pkg":"*"}}')
file(importer, 'export {}')
// A workspace-style link: the importer resolves `pkg` from its own node_modules to the source directory.
mkdirSync(join(root, 'importer', 'node_modules'), { recursive: true })
const link = join(root, 'importer', 'node_modules', 'pkg')
symlinkSync(dir, link, process.platform === 'win32' ? 'junction' : 'dir')
for (const name of ['a', 'b']) {
file(join(dir, `${name}.mjs`), `export const marker = '${name}'`)
file(join(dir, `${name}.cjs`), `module.exports = { marker: '${name}' }`)
}
const manifests = new PackageManifests()
cleanup.push(() => { manifests.dispose() })
const invalidate = (path: string) => Promise.resolve().then(manifests.invalidate.bind(manifests, path))
return { root, dir, manifest, importer, importerURL: pathToFileURL(importer).href, manifests, invalidate }
}
function resolveEsm(specifier: string, parentURL: string): string {
const loader = ModuleLoader.fromInternal()!
return (loader.version === 'v2'
? loader.resolveSync(parentURL, { specifier, attributes: {} })
: loader.resolveSync(specifier, parentURL, {})).url
}
describe('package manifest invalidation', { concurrent: false }, () => {
it('preserves filename-based resolve hooks and node-prefixed builtins after invalidation', async () => {
const f = fixture()
file(f.manifest, '{}')
const importerRequire = createRequire(f.importer)
const nativeFs = importerRequire('node:fs') as typeof import('node:fs')
const missingBuiltin = 'node:dsh-hmr-missing'
const missingFailure = outcome(() => importerRequire(missingBuiltin))
expect(missingFailure).toHaveProperty('code', 'ERR_UNKNOWN_BUILTIN_MODULE')
const target = join(f.dir, 'a.cjs')
const specifier = join(f.dir, 'alias.cjs')
file(specifier, 'module.exports = { marker: "alias" }')
cleanup.push(() => { Reflect.deleteProperty(importerRequire.cache, target) })
let hooks: ReturnType<typeof registerHooks> | undefined = registerHooks({
resolve(request, context, nextResolve) {
const resolved = nextResolve(request, context)
return resolved.url === pathToFileURL(specifier).href ? { ...resolved, url: pathToFileURL(target).href } : resolved
},
})
cleanup.push(() => { hooks?.deregister() })
try {
const original = importerRequire(specifier) as { marker: string }
expect(original).toEqual({ marker: 'a' })
await f.invalidate(f.manifest)
expect(importerRequire(specifier)).toBe(original)
} finally {
hooks.deregister()
hooks = undefined
}
const descriptor = Object.getOwnPropertyDescriptor(importerRequire.cache, 'fs')
const restore = () => {
if (descriptor) Object.defineProperty(importerRequire.cache, 'fs', descriptor)
else Reflect.deleteProperty(importerRequire.cache, 'fs')
}
cleanup.push(restore)
const fake = new NodeModule('fs')
const fakeExports = { marker: 'fake fs' }
fake.exports = fakeExports
try {
importerRequire.cache.fs = fake
expect(importerRequire('fs')).toBe(fakeExports)
expect(importerRequire('node:fs')).toBe(nativeFs)
expect(outcome(() => importerRequire(missingBuiltin))).toEqual(missingFailure)
} finally {
restore()
}
})
it('loads changed CommonJS exports without evicting previously loaded modules', async () => {
const f = fixture()
const a = join(f.dir, 'a.cjs')
const b = join(f.dir, 'b.cjs')
const unrelated = join(f.root, 'unrelated.cjs')
file(unrelated, 'module.exports = { marker: "unrelated" }')
file(f.manifest, JSON.stringify({ name: 'pkg', exports: './a.cjs' }))
const importerRequire = createRequire(f.importer)
cleanup.push(() => {
for (const path of [a, b, unrelated]) Reflect.deleteProperty(importerRequire.cache, path)
})
const original = importerRequire('pkg') as { marker: string }
const other = importerRequire(unrelated) as { marker: string }
const originalModule = importerRequire.cache[a]
const otherModule = importerRequire.cache[unrelated]
expect(original).toEqual({ marker: 'a' })
expect(originalModule).toBeDefined()
expect(otherModule).toBeDefined()
file(f.manifest, JSON.stringify({ name: 'pkg', exports: './b.cjs' }))
await f.invalidate(f.manifest)
expect(importerRequire.resolve('pkg')).toBe(b)
const current = importerRequire('pkg') as { marker: string }
expect(importerRequire.cache[a]).toBe(originalModule)
expect(importerRequire.cache[unrelated]).toBe(otherModule)
expect(original).toEqual({ marker: 'a' })
expect(importerRequire(a)).toBe(original)
expect(importerRequire(unrelated)).toBe(other)
expect(current).toEqual({ marker: 'b' })
})
it.each(['symlink', 'main'] as const)('refreshes a cached external %s entry after its manifest changes', async (entry) => {
const f = fixture()
const external = join(f.root, 'packages', 'outside', 'entry.mjs')
file(external, 'export {}')
if (entry === 'symlink') {
symlinkSync(dirname(external), join(f.dir, 'linked'), process.platform === 'win32' ? 'junction' : 'dir')
file(f.manifest, JSON.stringify({ name: 'pkg', exports: './linked/entry.mjs' }))
} else {
file(f.manifest, JSON.stringify({ name: 'pkg', main: external }))
}
const importerRequire = createRequire(f.importer)
expect(importerRequire.resolve('pkg')).toBe(external)
expect(resolveEsm('pkg', f.importerURL)).toBe(pathToFileURL(external).href)
file(f.manifest, JSON.stringify({ name: 'pkg', main: './b.mjs', exports: './b.mjs' }))
await f.invalidate(f.manifest)
expect.soft(importerRequire.resolve('pkg')).toBe(join(f.dir, 'b.mjs'))
expect(resolveEsm('pkg', f.importerURL)).toBe(pathToFileURL(join(f.dir, 'b.mjs')).href)
})
it('resolves changed exports and main only after the manifest is invalidated', () => {
const f = fixture()
file(f.manifest, JSON.stringify({ name: 'pkg', exports: { import: './a.mjs', require: './a.cjs' } }))
const importerRequire = createRequire(f.importer)
expect(resolveEsm('pkg', f.importerURL)).toBe(pathToFileURL(join(f.dir, 'a.mjs')).href)
expect(importerRequire.resolve('pkg')).toBe(join(f.dir, 'a.cjs'))
file(f.manifest, JSON.stringify({ name: 'pkg', exports: { import: './b.mjs', require: './b.cjs' } }))
expect(resolveEsm('pkg', f.importerURL)).toBe(pathToFileURL(join(f.dir, 'a.mjs')).href)
f.manifests.invalidate(f.manifest)
expect(resolveEsm('pkg', f.importerURL)).toBe(pathToFileURL(join(f.dir, 'b.mjs')).href)
expect(importerRequire.resolve('pkg')).toBe(join(f.dir, 'b.cjs'))
file(f.manifest, JSON.stringify({ name: 'pkg', main: './a.cjs' }))
f.manifests.invalidate(f.manifest)
expect(importerRequire.resolve('pkg')).toBe(join(f.dir, 'a.cjs'))
})
it('updates package type, scope, and nearest-package reads inside the directory', () => {
const f = fixture()
file(f.manifest, JSON.stringify({ name: 'pkg', type: 'commonjs' }))
const source = join(f.dir, 'lib', 'entry.js')
file(source, 'export {}')
expect(formats.defaultGetFormatWithoutErrors(pathToFileURL(source))).toBe('commonjs')
expect(reader.getNearestParentPackageJSON(source)?.data.type).toBe('commonjs')
file(f.manifest, JSON.stringify({ name: 'pkg', type: 'module' }))
f.manifests.invalidate(f.manifest)
expect(formats.defaultGetFormatWithoutErrors(pathToFileURL(source))).toBe('module')
expect(reader.getNearestParentPackageJSON(source)?.data.type).toBe('module')
expect(reader.getPackageScopeConfig(pathToFileURL(source).href)).toMatchObject({
exists: true, type: 'module', pjsonPath: toNamespacedPath(f.manifest),
})
})
it('resolves changed package imports from inside the directory', () => {
const f = fixture()
file(f.manifest, JSON.stringify({ name: 'pkg', type: 'module', imports: { '#target': './a.mjs' } }))
const parent = pathToFileURL(join(f.dir, 'inside.mjs')).href
file(join(f.dir, 'inside.mjs'), 'export {}')
expect(resolveEsm('#target', parent)).toBe(pathToFileURL(join(f.dir, 'a.mjs')).href)
file(f.manifest, JSON.stringify({ name: 'pkg', type: 'module', imports: { '#target': './b.mjs' } }))
f.manifests.invalidate(f.manifest)
expect(resolveEsm('#target', parent)).toBe(pathToFileURL(join(f.dir, 'b.mjs')).href)
})
it('reports an invalid manifest with the native error and recovers after it is fixed', () => {
const f = fixture()
file(f.manifest, JSON.stringify({ name: 'pkg', exports: './a.mjs' }))
expect(resolveEsm('pkg', f.importerURL)).toBe(pathToFileURL(join(f.dir, 'a.mjs')).href)
file(f.manifest, '{')
f.manifests.invalidate(f.manifest)
expect(() => reader.read(f.manifest)).toThrow(expect.objectContaining({ code: 'ERR_INVALID_PACKAGE_CONFIG' }))
file(f.manifest, JSON.stringify({ name: 'pkg', exports: './b.mjs' }))
f.manifests.invalidate(f.manifest)
expect(resolveEsm('pkg', f.importerURL)).toBe(pathToFileURL(join(f.dir, 'b.mjs')).href)
})
it('leaves other package directories to Node and restores native readers on disposal', () => {
const f = fixture()
const other = join(f.root, 'packages', 'other', 'package.json')
file(other, JSON.stringify({ name: 'other', type: 'commonjs' }))
file(f.manifest, JSON.stringify({ name: 'pkg', type: 'commonjs' }))
expect(reader.read(other).type).toBe('commonjs')
const original = binding.readPackageJSON
f.manifests.invalidate(f.manifest)
file(other, JSON.stringify({ name: 'other', type: 'module' }))
expect(reader.read(other).type).toBe('commonjs')
expect(binding.readPackageJSON).not.toBe(original)
f.manifests.dispose()
expect(binding.readPackageJSON).toBe(original)
})
it('reports invalid JSON with its non-file ESM base after invalidation', async () => {
const f = fixture()
file(f.manifest, '{')
await f.invalidate(f.manifest)
const base = 'data:text/javascript,export{}'
const failure = outcome(() => reader.read(f.manifest, { isESM: true, base, specifier: 'pkg' }))
expect(failure).toHaveProperty('code', 'ERR_INVALID_PACKAGE_CONFIG')
expect(failure).toHaveProperty('message', expect.stringContaining(base))
})
})
function outcome(run: () => unknown): unknown {
try { return run() } catch (error) {
const failure = error as Error & { code?: string }
return { code: failure.code, message: failure.message }
}
}
describe('package manifest reader parity', { concurrent: false }, () => {
it('terminates scope lookup after invalidating the filesystem root without modifying it', async () => {
const f = fixture()
const child = spawn(process.execPath, [
'--expose-internals',
fileURLToPath(new URL('./fixtures/package-root-scope.mjs', import.meta.url)),
f.root,
], { stdio: ['ignore', 'pipe', 'pipe'] })
const completed = new Promise<{ code: number | null; signal: NodeJS.Signals | null }>((resolve, reject) => {
child.once('error', reject)
child.once('close', (code, signal) => { resolve({ code, signal }) })
})
cleanup.push(async () => {
if (child.exitCode === null && child.signalCode === null) child.kill()
await completed
})
let stdout = '', stderr = ''
child.stdout.setEncoding('utf8').on('data', (chunk: string) => { stdout += chunk })
child.stderr.setEncoding('utf8').on('data', (chunk: string) => { stderr += chunk })
const result = await completed
expect(result.signal, stderr).toBeNull()
expect(result.code, stderr).toBe(0)
expect(stdout).toBe('scope traversal completed\n')
})
it('reads valid JSON with a non-file ESM base like Node', async () => {
const f = fixture()
file(f.manifest, '{}')
const options = { isESM: true, base: 'data:text/javascript,export{}', specifier: 'pkg' }
const expected = outcome(() => reader.read(f.manifest, options))
await f.invalidate(f.manifest)
expect(outcome(() => reader.read(f.manifest, options))).toEqual(expected)
})
it('refreshes a cached real directory after a package link moves', async () => {
const f = fixture()
const other = join(f.root, 'packages', 'other')
const link = join(f.root, 'importer', 'node_modules', 'pkg')
const linkedManifest = join(link, 'package.json')
file(f.manifest, '{"type":"commonjs"}')
file(join(other, 'package.json'), '{"type":"commonjs"}')
unlinkSync(link)
symlinkSync(other, link, process.platform === 'win32' ? 'junction' : 'dir')
await f.invalidate(f.manifest)
expect(reader.read(linkedManifest).type).toBe('commonjs')
unlinkSync(link)
symlinkSync(f.dir, link, process.platform === 'win32' ? 'junction' : 'dir')
file(f.manifest, '{"type":"module"}')
await f.invalidate(f.manifest)
expect(reader.read(linkedManifest).type).toBe('module')
})
it.each([
'{}',
'{"name":"pkg","main":"./a.cjs","type":"commonjs"}',
'{"name":"pkg","exports":{"import":"./a.mjs","require":"./a.cjs"}}',
'{"name":"pkg","exports":false,"main":"./a.cjs"}',
'{"name":"pkg","imports":{"#item":"./a.mjs"},"type":"module"}',
'{"name":"pkg","exports":"./a.mjs"}',
'{', 'null', '[]', '1', '{"name":1}', '{"type":1}', '{"type":"other"}',
'{"name":"\\ud800"}', '{"main":"\\ud800"}', '{"type":"\\ud800"}', '{"exports":"\\ud800"}',
])('reads %s like the native reader', (source) => {
const f = fixture()
file(f.manifest, source)
const expected = outcome(() => reader.read(f.manifest))
f.manifests.invalidate(f.manifest)
expect(outcome(() => reader.read(f.manifest))).toEqual(expected)
})
it('reports an invalid manifest found while importing like the native resolver', () => {
const f = fixture()
const errorOf = (run: () => unknown): Error => {
try { run() } catch (error) { return error as Error }
throw new Error('expected the lookup to fail')
}
file(f.manifest, '{')
const expected = errorOf(() => resolveEsm('pkg', f.importerURL))
f.manifests.invalidate(f.manifest)
const actual = errorOf(() => resolveEsm('pkg', f.importerURL))
expect([String(actual), Object.keys(actual)]).toEqual([String(expected), Object.keys(expected)])
expect(String(errorOf(() => reader.read(f.manifest)))).toBe(String(expected).replace(/ while importing .*(?=\.$)/u, '').replace(join(f.root, 'importer', 'node_modules', 'pkg'), f.dir))
})
it('reads invalid UTF-8 like the native reader', () => {
const f = fixture()
mkdirSync(f.dir, { recursive: true })
writeFileSync(f.manifest, Buffer.concat([Buffer.from('{"name":"'), Buffer.from([0xff]), Buffer.from('"}')]))
const expected = outcome(() => reader.read(f.manifest))
f.manifests.invalidate(f.manifest)
expect(outcome(() => reader.read(f.manifest))).toEqual(expected)
})
it('refreshes a changed CommonJS main after a cached lookup', () => {
const f = fixture()
file(f.manifest, JSON.stringify({ name: 'pkg', main: './a.cjs' }))
const importerRequire = createRequire(f.importer)
expect(importerRequire.resolve('pkg')).toBe(join(f.dir, 'a.cjs'))
file(f.manifest, JSON.stringify({ name: 'pkg', main: './b.cjs' }))
expect(importerRequire.resolve('pkg')).toBe(join(f.dir, 'a.cjs'))
f.manifests.invalidate(f.manifest)
expect(importerRequire.resolve('pkg')).toBe(join(f.dir, 'b.cjs'))
})
it('updates the package type of extensionless files', () => {
const f = fixture()
file(f.manifest, JSON.stringify({ name: 'pkg', type: 'commonjs' }))
const extensionless = pathToFileURL(join(f.dir, 'bin', 'tool'))
file(join(f.dir, 'bin', 'tool'), '')
const before = formats.defaultGetFormatWithoutErrors(extensionless)
file(f.manifest, JSON.stringify({ name: 'pkg', type: 'module' }))
f.manifests.invalidate(f.manifest)
expect(formats.defaultGetFormatWithoutErrors(extensionless)).not.toBe(before)
expect(formats.defaultGetFormatWithoutErrors(extensionless)).toBe('module')
})
it('continues to the enclosing scope after the manifest is deleted', () => {
const f = fixture()
const enclosing = join(f.root, 'packages', 'package.json')
file(enclosing, JSON.stringify({ name: 'packages', type: 'commonjs' }))
file(f.manifest, JSON.stringify({ name: 'pkg', type: 'module' }))
const source = join(f.dir, 'lib', 'entry.js')
file(source, '')
expect(reader.getPackageScopeConfig(pathToFileURL(source).href).type).toBe('module')
rmSync(f.manifest)
f.manifests.invalidate(f.manifest)
expect(reader.getPackageScopeConfig(pathToFileURL(source).href)).toMatchObject({
exists: true, type: 'commonjs', pjsonPath: toNamespacedPath(enclosing),
})
expect(formats.defaultGetFormatWithoutErrors(pathToFileURL(join(f.dir, 'bin', 'tool')))).toBe(
formats.defaultGetFormatWithoutErrors(pathToFileURL(join(f.root, 'packages', 'tool'))),
)
expect(reader.getNearestParentPackageJSON(source)?.path).toBe(toNamespacedPath(enclosing))
})
it('stops a scope lookup at a node_modules boundary like Node', () => {
const f = fixture()
file(f.manifest, JSON.stringify({ name: 'pkg', type: 'module' }))
const nested = join(f.dir, 'node_modules', 'dep', 'entry.js')
file(nested, '')
const expected = [
outcome(() => reader.getPackageScopeConfig(pathToFileURL(nested).href)),
outcome(() => reader.getNearestParentPackageJSON(nested)),
]
f.manifests.invalidate(f.manifest)
expect([
outcome(() => reader.getPackageScopeConfig(pathToFileURL(nested).href)),
outcome(() => reader.getNearestParentPackageJSON(nested)),
]).toEqual(expected)
})
it('answers missing directories and non-file URLs natively', () => {
const f = fixture()
file(f.manifest, '{}')
const missing = join(f.root, 'missing', 'package.json')
const nativeScope = binding.getPackageScopeConfig as (url: string) => unknown
const expected = [outcome(() => reader.read(missing)), outcome(() => nativeScope('node:fs'))]
f.manifests.invalidate(f.manifest)
expect([outcome(() => reader.read(missing)), outcome(() => (binding.getPackageScopeConfig as (url: string) => unknown)('node:fs'))])
.toEqual(expected)
})
it('falls back to the builtin addon for internals that --expose-internals does not expose', () => {
const f = fixture()
file(f.manifest, JSON.stringify({ name: 'pkg', main: './a.cjs' }))
const importerRequire = createRequire(f.importer)
expect(importerRequire.resolve('pkg')).toBe(join(f.dir, 'a.cjs'))
const argv = [...process.execArgv]
process.execArgv.push('--expose-internals')
try {
file(f.manifest, JSON.stringify({ name: 'pkg', main: './b.cjs' }))
f.manifests.invalidate(f.manifest)
} finally {
process.execArgv.splice(0, process.execArgv.length, ...argv)
}
expect(importerRequire.resolve('pkg')).toBe(join(f.dir, 'b.cjs'))
})
it('keeps readers replaced after installation when disposed', async () => {
const f = fixture()
file(f.manifest, '{}')
const previousRead = binding.readPackageJSON
const previousScope = binding.getPackageScopeConfig
const previousType = binding.getPackageType
const previousNearest = reader.getNearestParentPackageJSON
try {
await f.invalidate(f.manifest)
const later = () => undefined
binding.readPackageJSON = later
reader.getNearestParentPackageJSON = later
f.manifests.dispose()
expect(binding.readPackageJSON).toBe(later)
expect(reader.getNearestParentPackageJSON).toBe(later)
expect(binding.getPackageScopeConfig).toBe(previousScope)
expect(binding.getPackageType).toBe(previousType)
} finally {
f.manifests.dispose()
binding.readPackageJSON = previousRead
binding.getPackageScopeConfig = previousScope
binding.getPackageType = previousType
reader.getNearestParentPackageJSON = previousNearest
}
})
})
@@ -24,8 +24,8 @@
en: ca75c51c89c2c9b0
zh: 64715dc7e6d02f71
/deepseek-ai-dsh-experimental-webworker-runtime/known-limitations-and-deferred-work:
en: 6fe586817a42fdad
zh: c19ab678d61e82d3
en: dd2469d07c86127a
zh: efa1be38e57bdd1e
/deepseek-ai-dsh-experimental-webworker-runtime/known-limitations-and-deferred-work/dev-note:
en: e67f633dc6fe7773
zh: e88aeb617ffb74ea
@@ -53,6 +53,7 @@ None; this package neither assembles nor sends a provider request.
- **The worker composition writes plaintext session logs** (`compression: 'none'` boot patch): it carries no Zstandard codec, so exported logs are `.jsonl`, never `.jsonl.zstd`.
- **`node:dns/promises`, `node:vm`, `node:net`, `node:sqlite`, `node:worker_threads` are structural stubs**: every call reports its refusal on the console and throws. Rows needing native DNS, a real process, or realm isolation cannot run here.
- **`node:assert/strict` supports truthiness assertions only**: for falsy values, the callable default and `ok` throw `ERR_ASSERTION` with the supplied message or generic text, or rethrow a supplied Error unchanged. Other assertion APIs are absent.
- **Desktop product telemetry is unavailable**: `got.post` reports an explicit worker-host refusal. Got and its Node HTTP dependencies are excluded from the browser image; Desktop reporting remains disabled in the preview composition.
- **Host package commands are unavailable**: `execa` reports an explicit worker-host refusal; the preview cannot run pnpm, install plugins, or install native dependencies.
- **PTC Node programs are unavailable**: the process shim exposes `/dsh/bin/node` as its executable identity so the provider can activate, but the Worker has neither a Node executable nor `stripTypeScriptTypes`. Program execution fails before launching a child.
@@ -53,6 +53,7 @@ kind: "package-library"
- **worker 组合写明文会话日志**(`compression: 'none'` boot patch):不带 Zstandard 编解码器,导出日志是 `.jsonl`,不会是 `.jsonl.zstd`。
- **`node:dns/promises`、`node:vm`、`node:net`、`node:sqlite`、`node:worker_threads` 是结构化 stub**:每次调用在 console 报告拒绝并抛出。需要原生 DNS、真进程或真 realm 隔离的行在此无法运行。
- **`node:assert/strict` 仅支持真值断言**:对于假值,可调用的默认导出和 `ok` 使用传入的消息或通用文本抛出 `ERR_ASSERTION`,或原样抛出传入的 Error。其他断言 API 未提供。
- **桌面产品埋点不可用**:`got.post` 明确报告 worker host 不支持该调用。浏览器镜像不包含 Got 及其 Node HTTP 依赖;预览组合不启用桌面上报。
- **宿主包管理命令不可用**:`execa` 明确报告 worker host 不支持该调用;预览无法运行 pnpm、安装插件或安装原生依赖。
- **PTC Node 程序不可用**:process shim 用 `/dsh/bin/node` 表示可执行文件身份,使 provider 能够激活,但 Worker 既没有 Node 可执行文件,也没有 `stripTypeScriptTypes`。程序执行会在启动子进程前失败。
@@ -44,6 +44,8 @@ export const MODULE_PROXIES: Record<string, string> = {
// Sync-stack AsyncLocalStorage semantics.
'node:async_hooks': './node/builtin_modules/implemented/async_hooks.ts',
// Real implementations over browser primitives.
'node:assert/strict': './node/builtin_modules/implemented/assert/strict.ts',
'assert/strict': './node/builtin_modules/implemented/assert/strict.ts',
'node:util': './node/builtin_modules/implemented/util.ts',
'node:util/types': './node/builtin_modules/implemented/util/types.ts',
'node:events': './node/builtin_modules/implemented/events.ts',
@@ -0,0 +1,28 @@
/** Truthiness assertions for Worker modules importing `node:assert/strict`. */
/**
* Require a truthy value; falsy values throw `ERR_ASSERTION` or the supplied Error.
* @param value - Value to test.
* @param message - Failure message or Error to throw unchanged; omitted messages use generic text.
* @returns Nothing; successful completion narrows the value to a truthy value.
*/
export function ok(value: unknown, message?: string | Error): asserts value {
if (value) return
if (message instanceof Error) throw message
throw Object.assign(new Error(message ?? 'The expression evaluated to a falsy value.'), {
name: 'AssertionError',
code: 'ERR_ASSERTION',
actual: value,
expected: true,
operator: '==',
generatedMessage: message === undefined,
})
}
/** CommonJS interop marker for lowered ESM default imports. */
export const __esModule = true
/** Callable truthiness assertion with the identical `ok` method; other assertion APIs are absent. */
const assert: typeof ok & { ok: typeof ok } = Object.assign(ok, { ok })
export default assert
@@ -20,7 +20,7 @@ export function createRequire(base: string | URL): NodeRequire {
/** Builtin specifiers the module proxy table answers (without the `node:` prefix). */
export const builtinModules = [
'assert', 'async_hooks', 'buffer', 'child_process', 'crypto', 'events', 'fs', 'http', 'module',
'assert', 'assert/strict', 'async_hooks', 'buffer', 'child_process', 'crypto', 'events', 'fs', 'http', 'module',
'net', 'os', 'path', 'process', 'stream', 'tty', 'url', 'util', 'worker_threads',
]
@@ -21,6 +21,7 @@
* other import. Deferring a shim's own start-up cost therefore belongs inside
* that shim, on the path that first needs it.
*/
import * as nodeAssertStrict from './builtin_modules/implemented/assert/strict.ts'
import * as nodeAsyncHooks from './builtin_modules/implemented/async_hooks.ts'
import * as nodeBuffer from './builtin_modules/implemented/buffer.ts'
import * as nodeCrypto from './builtin_modules/implemented/crypto.ts'
@@ -61,6 +62,7 @@ import type { StaticModuleFactory } from '../module-system/module-loader.ts'
/** Builtin modules, keyed with and without the `node:` prefix. */
const BUILTINS: Record<string, StaticModuleFactory> = {
'assert/strict': () => nodeAssertStrict,
async_hooks: () => nodeAsyncHooks,
buffer: () => nodeBuffer,
child_process: () => nodeChildProcess,
@@ -0,0 +1,78 @@
/** Strict assertion resolution and execution through the Worker image loader. */
import { describe, expect, it } from 'vitest'
import { lowerModuleSource } from '../../src/compile/transform.ts'
import { MODULE_PROXIES } from '../../src/module-proxies.ts'
import { WorkerModuleLoader } from '../../src/module-system/module-loader.ts'
import { createNodeBuiltins } from '../../src/node/builtins.ts'
import assert, { ok } from '../../src/node/builtin_modules/implemented/assert/strict.ts'
import { isBuiltin } from '../../src/node/builtin_modules/implemented/module.ts'
import { MemoryVfs } from '../../src/storage/memory.ts'
describe('Worker strict assertions', () => {
it.each([true, 1, 'present', {}, []])('accepts truthy value %j', (value) => {
expect(() => { assert(value) }).not.toThrow()
})
it.each([false, 0, '', null, undefined, NaN])('rejects falsy value %j with assertion fields', (value) => {
expect(() => { assert(value, 'HMR pending module is missing') }).toThrow(expect.objectContaining({
name: 'AssertionError',
code: 'ERR_ASSERTION',
message: 'HMR pending module is missing',
actual: value,
expected: true,
operator: '==',
generatedMessage: false,
}))
})
it('generates a message when none is supplied', () => {
expect(() => { assert(false) }).toThrow(expect.objectContaining({
message: 'The expression evaluated to a falsy value.',
generatedMessage: true,
}))
})
it('preserves an explicitly empty message', () => {
expect(() => { assert(false, '') }).toThrow(expect.objectContaining({ message: '', generatedMessage: false }))
})
it('throws a supplied Error unchanged', () => {
const error = new Error('missing module')
expect.assertions(2)
try {
assert(false, error)
} catch (reason) {
expect(reason).toBe(error)
}
expect(() => { assert(true, error) }).not.toThrow()
})
it('shares the callable default and ok exports', () => {
expect(assert).toBe(ok)
expect(assert.ok).toBe(ok)
})
it.each(['node:assert/strict', 'assert/strict'])('resolves and executes a lowered default import from %s', (specifier) => {
const vfs = new MemoryVfs()
vfs.seedDirectory('/dsh')
const loader = new WorkerModuleLoader({ vfs, root: '/dsh', staticModules: createNodeBuiltins() })
const transformed = lowerModuleSource({
filename: '/dsh/probe.js',
source: `import assert, { ok } from '${specifier}';
assert({ getNamespace() {} }, 'HMR pending module is missing');
ok(true);
export const same = assert === ok && assert.ok === ok;
export function reject() { assert(undefined, 'HMR pending module is missing'); }`,
})
expect(transformed.moduleRequests).toContain(specifier)
expect(loader.resolve(specifier, '/dsh')).toMatchObject({ kind: 'static' })
expect(isBuiltin(specifier)).toBe(true)
expect(MODULE_PROXIES[specifier]).toBe('./node/builtin_modules/implemented/assert/strict.ts')
vfs.writeFileSync('/dsh/probe.js', transformed.code)
const require = loader.createRequire('/dsh/')
expect(require('assert/strict')).toBe(require('node:assert/strict'))
const probe = require('./probe.js') as { same: boolean; reject: () => void }
expect(probe.same).toBe(true)
expect(probe.reject).toThrow(expect.objectContaining({ code: 'ERR_ASSERTION', message: 'HMR pending module is missing' }))
})
})
@@ -6164,7 +6164,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'Reload',
declaration: 'export interface Reload {\n filename: string;\n runtime?: Plugin.Runtime | undefined;\n}',
declaration: 'export interface Reload {\n filename: string;\n modules: ReloadModules;\n runtime?: Plugin.Runtime | undefined;\n}',
},
{
name: 'RemoteError',
+3
View File
@@ -1804,6 +1804,9 @@ importers:
chokidar:
specifier: ^4.0.3
version: 4.0.3
node-addon-require-builtin:
specifier: ^0.1.6
version: 0.1.6
picomatch:
specifier: ^4.0.3
version: 4.0.4
+2
View File
@@ -56,6 +56,8 @@ Keep this log exhaustive — every divergence from upstream must be listed.
22. **Volatile config across `cosmokit`, `schemastery`, `cordis`, and `loader`**: Cosmokit supplies immutable references and framework-owned commits; `deepEqual` treats two references as equal, propagates strict comparison into arrays, compares every array slot with holes treated as undefined, handles cycles conservatively, compares URLs by normalized href, and compares other opaque objects by identity in strict mode. Schemastery adds `.volatile()`, optional/default inference, placement validation without eager lazy-schema expansion (a lazy inner is validated once when built), metadata-free serialized node compatibility, serialized metadata, reference-aware simplification, unsupported volatile values reported as `ValidationError` with their path, and `.default()` accepting the raw input type `Partial<S>` because a volatile field makes the output type carry references. Loader's save path calls `simplify` with its schema receiver (`unparse.call`), an independent fix for the unbound-`this` call. Cordis exports consumer types without changing Fiber lifecycle. Loader compares raw options strictly in `Entry.update`; for a config-only change on an active fiber it calls the pure `equalExceptVolatile` function in `config/diff.ts`, which narrows the Standard Schema to Schemastery's `Schema` type (a new type-only `@deepseek-ai/schemastery` dependency of `loader`) and recursively skips fixed volatile paths using schema metadata without executing config hooks or validation. Missing or null object nodes compare using their declared defaults, including nodes without volatile descendants; schema backedges retain raw comparison. Entry passes its actual schema and preserves ordinary context and custom-update lifecycles. For a volatile-only change, `Entry._commitVolatile` re-parses the raw config through the fiber's `internal/config` hook and `resolveConfig`, compares ordinary effective values, commits references and emits the instance-local `loader/volatile-update` declared in `loader/src/index.ts`; a changed ordinary effective value falls back to the ordinary remount, and an invalid candidate is logged while the raw config stays retained for the next activation. Cosmokit provides reference enumeration and commits, and `simplify` is called with its schema receiver. The package READMEs document these additions; schema tests live in `scripts/volatile-config.spec.ts`, Loader runtime and real-file tests in `scripts/loader-volatile-update.spec.ts`, and code replacement tests in the HMR module suite.
23. **`loader/src/config/entry.ts` module identity**: retains each entry's raw import result as `moduleNamespace`, before plugin export normalization. HMR matches that object to Node's cached module namespace and updates it after a successful reload. `EntryTree.import()` keeps its existing arguments and return value; `loader/README.md` documents the additive field.
## Sync procedure
To update a vendored package from upstream:
+1
View File
@@ -41,6 +41,7 @@ root.loader.update(id, { config: { enabled: false } })
| `loader.update(id, options, parent?, position?)` | Update, move, and restart an entry. |
| `loader.remove(id)` | Stop and delete an entry. |
| `loader.resolve(id)` | Resolve an entry by id, including nested `a:b` ids. |
| `loader.resolve(id).moduleNamespace` | Raw import result before export normalization; HMR updates it after a successful reload. |
| `loader.resolveGroup(id)` | Resolve the root group or a nested group. |
| `loader.await()` | Wait for pending entry imports and fiber reloads. |
| `loader.locate(fiber?)` | Return the loader entry id that owns a fiber. |
+6 -3
View File
@@ -45,6 +45,8 @@ export class Entry {
public ctx: Context
public fiber?: Fiber
/** Raw import result before export normalization; HMR updates it after a successful reload. */
public moduleNamespace: unknown
public parent!: EntryGroup
// safety: call `entry.update()` immediately after creating an entry
public options = {} as EntryOptions
@@ -219,18 +221,19 @@ export class Entry {
}
private async _init() {
let exports: any
let moduleNamespace: unknown
try {
exports = await this.parent.tree.import(this.options.name, this.getOuterStack)
moduleNamespace = await this.parent.tree.import(this.options.name, this.getOuterStack)
} catch (error) {
this.ctx.logger.error(error)
return
} finally {
this._initTask = undefined
}
const plugin = this.loader.unwrapExports(exports)
const plugin = this.loader.unwrapExports(moduleNamespace)
this._patchContext([])
this.loader.showLog(this, 'apply')
this.fiber = this.ctx.registry.plugin(plugin, this.options.config, this.getOuterStack).ctx.fiber
this.moduleNamespace = moduleNamespace
}
}