fix(storage): propagate unknown-state through domains and clients

This commit is contained in:
Chinesezjc committed 2026-10-22 18:31:12 +08:00
1 parent 0397bd3f04
commit 3c7cfb8cf8
25 files changed
+225 -46

No files matched your search

+2 -2
View File
@@ -27,8 +27,8 @@
en: 93798ad69f74ddd5
zh: 634126ad8accc356
/host-wide-schedule/web-catalog:
en: a4f9de4d4f88f018
zh: 228a2e8fcc726f4c
en: 4d8f28a57062ce02
zh: d8fe886d103b3d97
/host-wide-schedule/timing-edits:
en: ce5d2095a3ad2e82
zh: 167b3715f8eaffc1
+3 -1
View File
@@ -345,7 +345,7 @@ type ScheduleCatalogEntry = ScheduleRecord & {
}
```
The Remote `schedule.list({ sessionId })`, model `schedule_list`, and Session-header catalog return only active tasks. A model view's derived timing `state` remains distinct from stored lifecycle `status`. Deletion uses `schedule.delete({ sessionId, id })` with the entry's original binding; a mismatched binding returns not found. Model tools supply the current Agent's Session, whereas the global user interface supplies the selected task's binding. The binding check alone does not establish caller authorization. `schedule.deleteInactive()` removes every ended (inactive) task across Sessions, with the saved delivery records each row carries; on success it answers `{ deleted }` with the count it removed, and a failed batch rejects with nothing removed and nothing announced. The Automation tasks page's bulk cleanup uses it. An ended task never arms a timer, so the removal recomputes no target. The payload-free `schedule/changed` event invalidates client lists; reconnecting clients fetch current state again.
The Remote `schedule.list({ sessionId })`, model `schedule_list`, and Session-header catalog return only active tasks. A model view's derived timing `state` remains distinct from stored lifecycle `status`. Deletion uses `schedule.delete({ sessionId, id })` with the entry's original binding; a mismatched binding returns not found. Model tools supply the current Agent's Session, whereas the global user interface supplies the selected task's binding. The binding check alone does not establish caller authorization. `schedule.deleteInactive()` removes every ended (inactive) task across Sessions, with the saved delivery records each row carries; on success it answers `{ deleted }` with the count it removed; a normal failed batch rejects with nothing removed, and an `unknown-state` failure may have removed rows, so the service announces `schedule/changed` before rejecting and clients re-read. The Automation tasks page's bulk cleanup uses it. An ended task never arms a timer, so the removal recomputes no target. The payload-free `schedule/changed` event invalidates client lists; reconnecting clients fetch current state again.
The shipped Web composition mounts `ui-schedule` with the Host capability. The [client package](../../packages/client/ui-schedule/README.md) owns the catalog, empty state, and deletion controls. The page separately filters all, active, and inactive tasks, retains inactive details and an original-Session control in the detail tab strip, and requires explicit confirmed deletion. The Rules and Records tabs separate task settings from lazily paged saved receipts. Neither a receipt nor inactive status confirms model execution.
@@ -524,6 +524,8 @@ async create(sessionId: SessionId, request: ScheduleCreateRequest, signal?: Abor
* slot, so a delivery ordered behind this call ends its task after the
* enumeration rather than during it. The batch is one durable write: a failure
* removes nothing and rejects, so the caller never receives a partial removal.
* A backend that reports `unknown-state` may have removed rows, so the service
* announces the change before rejecting and clients re-read the catalog.
* @returns Count of removed rows; zero when the Host held no ended task.
*/
@Remote('deleteInactive') async deleteInactive(): Promise<ScheduleDeleteInactiveResult>
+3 -1
View File
@@ -345,7 +345,7 @@ type ScheduleCatalogEntry = ScheduleRecord & {
}
```
Remote 方法 `schedule.list({ sessionId })`、模型 `schedule_list` 和 Session 页头目录仅返回活动任务。模型视图派生的时间 `state` 与存储的生命周期 `status` 仍是不同概念。删除通过 `schedule.delete({ sessionId, id })` 使用条目的原始绑定;绑定不匹配时返回未找到。模型工具传入当前 Agent 的 Session,全局用户界面则传入所选任务的绑定。仅校验绑定并不构成调用者鉴权。`schedule.deleteInactive()` 删除跨会话的全部未运行(已结束)任务,连同每行携带的已保存的发送记录;成功时以它移除的计数答复 `{ deleted }`;该批删除是一次持久写入,失败时不删除任何行,也不发通知,调用以拒绝结束。自动化任务页面的批量清理使用该方法。未运行任务从不会安排定时器,因此该删除不重算任何目标。无 payload 的 `schedule/changed` 事件通知客户端刷新列表;重新连接后,客户端再次获取当前状态。
Remote 方法 `schedule.list({ sessionId })`、模型 `schedule_list` 和 Session 页头目录仅返回活动任务。模型视图派生的时间 `state` 与存储的生命周期 `status` 仍是不同概念。删除通过 `schedule.delete({ sessionId, id })` 使用条目的原始绑定;绑定不匹配时返回未找到。模型工具传入当前 Agent 的 Session,全局用户界面则传入所选任务的绑定。仅校验绑定并不构成调用者鉴权。`schedule.deleteInactive()` 删除跨会话的全部未运行(已结束)任务,连同每行携带的已保存的发送记录;成功时以它移除的计数答复 `{ deleted }`;普通失败时不删除任何行,后端报告 `unknown-state` 时可能已移除部分行,服务会先发出 `schedule/changed` 再拒绝,客户端重新读取。自动化任务页面的批量清理使用该方法。未运行任务从不会安排定时器,因此该删除不重算任何目标。无 payload 的 `schedule/changed` 事件通知客户端刷新列表;重新连接后,客户端再次获取当前状态。
随发行版交付的 Web 组合把 `ui-schedule` 与宿主能力一起挂载。[客户端包](../../packages/client/ui-schedule/README.zh.md) 负责目录、空状态与删除控件。页面单独筛选全部、活动和未运行任务,保留未运行任务详情及详情页签条内的原 Session 入口,并要求显式确认删除。“规则”和“任务运行记录”页签将任务设置与按需分页加载的已保存回执分开。回执与未运行状态均不确认模型执行。
@@ -524,6 +524,8 @@ async create(sessionId: SessionId, request: ScheduleCreateRequest, signal?: Abor
* slot, so a delivery ordered behind this call ends its task after the
* enumeration rather than during it. The batch is one durable write: a failure
* removes nothing and rejects, so the caller never receives a partial removal.
* A backend that reports `unknown-state` may have removed rows, so the service
* announces the change before rejecting and clients re-read the catalog.
* @returns Count of removed rows; zero when the Host held no ended task.
*/
@Remote('deleteInactive') async deleteInactive(): Promise<ScheduleDeleteInactiveResult>
+2 -2
View File
@@ -9,8 +9,8 @@
en: b11e06e7c0e0fd13
zh: 4585f26ff416cb41
/storage/the-backend-contract:
en: b83300f10cc822f2
zh: 61835181022a7ba6
en: e9427832bd1dd9c7
zh: 41176b1431ec965c
/storage/declaring-a-domain:
en: 775cce941500f01f
zh: 78acbc67f48574cb
+1 -1
View File
@@ -44,7 +44,7 @@ interface StorageBackend {
}
```
A backend owns one medium (a file-tree root, a database file) and exposes optional operation groups; `kv` is the only shipped group. `KvFacet.open(descriptor)` opens one named unit — `KvUnitDescriptor` carries the name, current format version, optional compatible record versions, table names, and whether a global singleton slot exists — and returns a `KvUnit` with `loadAll`, `putRecord`, `deleteRecord`, `deleteRecords`, `setGlobal`, and `close`. Unit and table names must match `UNIT_NAME_RE` (safe as a file name and as a SQL identifier segment); record keys are arbitrary strings that never reach file paths. A unit does not serialize concurrent writes — ordering belongs to the caller — but each single call is atomic on the medium and durable once resolved. `deleteRecords` applies a whole batch in one call: it resolves once every listed record is durably absent, and a rejection leaves the batch unapplied; a backend that cannot restore staged records rejects with `unknown-state` and then rejects every later operation on that unit. A `single` medium stamped with a different version rejects `version-mismatch`; a `per-record` document stamped outside the accepted set reads as absent. A medium that cannot be parsed as the unit rejects `malformed-medium`. [`backend.ts`](../../packages/storage/storage/src/backend.ts) is the normative clause-by-clause contract, and the shared conformance suite in [`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) checks every clause against each backend. The [json backend](../../packages/storage/storage-json/README.md) republishes one whole human-readable file per unit atomically; the [sqlite backend](../../packages/storage/storage-sqlite/README.md) stores one document per row in one database for frequently updated data.
A backend owns one medium (a file-tree root, a database file) and exposes optional operation groups; `kv` is the only shipped group. `KvFacet.open(descriptor)` opens one named unit — `KvUnitDescriptor` carries the name, current format version, optional compatible record versions, table names, and whether a global singleton slot exists — and returns a `KvUnit` with `loadAll`, `putRecord`, `deleteRecord`, `deleteRecords`, `setGlobal`, and `close`. Unit and table names must match `UNIT_NAME_RE` (safe as a file name and as a SQL identifier segment); record keys are arbitrary strings that never reach file paths. A unit does not serialize concurrent writes — ordering belongs to the caller — but each single call is atomic on the medium and durable once resolved. `deleteRecords` applies a whole batch in one call: it resolves once every listed record is absent from the readable state and the backend accepted the batch, and a rejection normally leaves the batch unapplied; a backend that cannot establish or restore the batch's state rejects with `unknown-state`, after which every later operation on that unit (except `close()`) rejects with the same error. A `single`-layout failure after the atomic replacement (a directory-fsync error) can leave the new file on the medium while the caller sees a rejection; the unit keeps its memory consistent and the next publish rewrites the file. A `single` medium stamped with a different version rejects `version-mismatch`; a `per-record` document stamped outside the accepted set reads as absent. A medium that cannot be parsed as the unit rejects `malformed-medium`. [`backend.ts`](../../packages/storage/storage/src/backend.ts) is the normative clause-by-clause contract, and the shared conformance suite in [`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) checks every clause against each backend. The [json backend](../../packages/storage/storage-json/README.md) republishes one whole human-readable file per unit atomically; the [sqlite backend](../../packages/storage/storage-sqlite/README.md) stores one document per row in one database for frequently updated data.
## Declaring a domain
+1 -1
View File
@@ -44,7 +44,7 @@ interface StorageBackend {
}
```
一个后端拥有一个介质(一棵文件树的根目录、一个数据库文件),并提供可选的操作组;`kv` 是唯一已交付的操作组。`KvFacet.open(descriptor)` 打开一个具名 unit——`KvUnitDescriptor` 携带名称、当前格式版本、可选的兼容记录版本、表名清单,以及是否存在全局单例 slot——并返回提供 `loadAll`、`putRecord`、`deleteRecord`、`deleteRecords`、`setGlobal` 和 `close` 的 `KvUnit`。unit 名与表名必须匹配 `UNIT_NAME_RE`(既可安全用作文件名,也可安全用作 SQL 标识符片段);记录键是任意字符串,绝不进入文件路径。unit 不对并发写入做串行化——顺序由调用方负责——但每次单独调用在介质上都是原子的,且 resolve 后即已持久。`deleteRecords` 以一次调用应用整批删除:所有列出的记录都持久缺失后才 resolve,拒绝时整批未应用;无法恢复暂存记录的批量删除会以 `unknown-state` 拒绝,并让该 unit 之后的所有操作都拒绝。`single` 介质上记录的版本不同时拒绝 `version-mismatch`;`per-record` 文档的版本在接受集合之外时读作不存在。无法按该 unit 解析的介质拒绝 `malformed-medium`。[`backend.ts`](../../packages/storage/storage/src/backend.ts) 是逐条款的规范性约定,[`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) 中的共享一致性套件会针对每个后端检查每项条款。[json 后端](../../packages/storage/storage-json/README.zh.md)以原子方式为每个 unit 整文件重新发布一份人类可读文件;[sqlite 后端](../../packages/storage/storage-sqlite/README.zh.md)在单个数据库中每行存储一份文档,用于频繁更新的数据。
一个后端拥有一个介质(一棵文件树的根目录、一个数据库文件),并提供可选的操作组;`kv` 是唯一已交付的操作组。`KvFacet.open(descriptor)` 打开一个具名 unit——`KvUnitDescriptor` 携带名称、当前格式版本、可选的兼容记录版本、表名清单,以及是否存在全局单例 slot——并返回提供 `loadAll`、`putRecord`、`deleteRecord`、`deleteRecords`、`setGlobal` 和 `close` 的 `KvUnit`。unit 名与表名必须匹配 `UNIT_NAME_RE`(既可安全用作文件名,也可安全用作 SQL 标识符片段);记录键是任意字符串,绝不进入文件路径。unit 不对并发写入做串行化——顺序由调用方负责——但每次单独调用在介质上都是原子的,且 resolve 后即已持久。`deleteRecords` 以一次调用应用整批删除:所有列出的记录都从可读状态消失且后端接受该批后才 resolve,拒绝时整批未应用;无法建立或恢复批次状态的批量删除会以 `unknown-state` 拒绝,之后该 unit 的所有操作(`close()` 除外)都以同一错误拒绝。`single` 布局在原子替换之后失败(目录 fsync 错误)时,新文件可能已落在介质上而调用方看到拒绝;unit 的内存保持一致,下一次发布会重写该文件。`single` 介质上记录的版本不同时拒绝 `version-mismatch`;`per-record` 文档的版本在接受集合之外时读作不存在。无法按该 unit 解析的介质拒绝 `malformed-medium`。[`backend.ts`](../../packages/storage/storage/src/backend.ts) 是逐条款的规范性约定,[`tests/contract.ts`](../../packages/storage/storage/tests/contract.ts) 中的共享一致性套件会针对每个后端检查每项条款。[json 后端](../../packages/storage/storage-json/README.zh.md)以原子方式为每个 unit 整文件重新发布一份人类可读文件;[sqlite 后端](../../packages/storage/storage-sqlite/README.zh.md)在单个数据库中每行存储一份文档,用于频繁更新的数据。
## 声明领域
@@ -2194,7 +2194,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
},
{
signature: '@Remote(\'deleteInactive\') async deleteInactive(): Promise<ScheduleDeleteInactiveResult>',
description: 'Delete every ended Host task, leaving active tasks armed.\n\nEnded tasks never schedule another delivery, so the removal changes no pending target and no timer is recomputed. Rows are enumerated and removed in one FIFO slot, so a delivery ordered behind this call ends its task after the enumeration rather than during it. The batch is one durable write: a failure removes nothing and rejects, so the caller never receives a partial removal.',
description: 'Delete every ended Host task, leaving active tasks armed.\n\nEnded tasks never schedule another delivery, so the removal changes no pending target and no timer is recomputed. Rows are enumerated and removed in one FIFO slot, so a delivery ordered behind this call ends its task after the enumeration rather than during it. The batch is one durable write: a failure removes nothing and rejects, so the caller never receives a partial removal. A backend that reports `unknown-state` may have removed rows, so the service announces the change before rejecting and clients re-read the catalog.',
parameters: [],
returns: 'Count of removed rows; zero when the Host held no ended task.',
},
+2 -2
View File
@@ -15,8 +15,8 @@
en: 23386452a66b2fbd
zh: 4a8d91b2f313fead
/deepseek-ai-dsh-schedule/use-this-package:
en: a4521f1f145b2dc7
zh: 690d626671678f93
en: e632295a37b7399d
zh: 3daae4e585e387c6
/deepseek-ai-dsh-schedule/understand-the-implementation:
en: 02848e6fd6317b2a
zh: 1807a423210f81ef
+1 -1
View File
@@ -44,7 +44,7 @@ Weekly input adds `weekdays`, a non-empty set of ISO weekday numbers from Monday
Cron input carries `expression` and `time_zone`. The expression is the standard five-field Vixie form `minute hour day-of-month month day-of-week`: minute 0-59, hour 0-23, day-of-month 1-31, month 1-12, and day-of-week 0-7 where both `0` and `7` mean Sunday. Every field accepts `*`, a single value, an `a-b` range, a `*/n` or `a-b/n` step with `n >= 1`, and a comma-separated list of those forms. `L`, `W`, `#`, `JAN`/`MON` names, `@daily`-style macros, six-field expressions, out-of-range values, inverted ranges, zero steps, and empty fields are rejected with `invalid_rule` and a message naming the offending field. Because the dialect has five fields, the smallest interval is one minute. Creation stores a canonical expression: repeated values collapse, adjacent values and ranges merge, a uniform step is written as `a-b/n` or, for a field that started with `*`, as a star-step (`*` for every value, otherwise the widest star-step walk plus any remaining values), a step of `1` is dropped, and Sunday is written as `0`. A field that did not start with `*` never becomes a star-step, so the day rule below survives storage. The durable decoder rejects a non-canonical stored expression, so the record always holds the canonical text. When either day-of-month or day-of-week is a star, a local date matches only when both fields match; when neither is a star, either field matching is enough. A field is a star when its text starts with `*`, independently of the values it matches, so a stepped star constrains alongside the other field instead of substituting for it. The first target is the first strictly future instant whose local date and time in that zone match, using the same gap skip and earlier-overlap rules as Daily. See [cron timing](../../../docs/subsystems/schedule.md#cron-wall-clock-input) for the full dialect and canonicalization rules.
A reminder is bound to the calling Agent's Session. The shared `schedule` Remote namespace exposes `catalog` for active and inactive Host tasks with their original Session ids, and `list`, `history`, `update`, and `delete` with an explicit Session id. It also exposes `deleteInactive`, which removes every inactive task across Sessions and answers how many rows it removed; an inactive task never arms a timer, so that removal recomputes no target, and the batch is one durable write, so a failure removes nothing and rejects instead of reporting a count. Remote `list` and model `schedule_list` return only active tasks. Catalog entries retain only the latest receipt in `lastDelivery`; catalog and list responses omit saved delivery history. None of these operations activates an Agent or reads Session logs. Explicit deletion stops future delivery, leaves the original Session and already queued messages intact, and removes the stored task row together with its saved delivery records: the task leaves `list` and `catalog`, never schedules again, and `history` answers `schedule_not_found` for the same `(sessionId, id)`.
A reminder is bound to the calling Agent's Session. The shared `schedule` Remote namespace exposes `catalog` for active and inactive Host tasks with their original Session ids, and `list`, `history`, `update`, and `delete` with an explicit Session id. It also exposes `deleteInactive`, which removes every inactive task across Sessions and answers how many rows it removed; an inactive task never arms a timer, so that removal recomputes no target, and the batch is one durable write, so a normal failure removes nothing and rejects instead of reporting a count; a backend `unknown-state` failure may have removed rows, so the service announces `schedule/changed` before rejecting and clients re-read. Remote `list` and model `schedule_list` return only active tasks. Catalog entries retain only the latest receipt in `lastDelivery`; catalog and list responses omit saved delivery history. None of these operations activates an Agent or reads Session logs. Explicit deletion stops future delivery, leaves the original Session and already queued messages intact, and removes the stored task row together with its saved delivery records: the task leaves `list` and `catalog`, never schedules again, and `history` answers `schedule_not_found` for the same `(sessionId, id)`.
Archiving a Session with active reminders is refused until they stop, and choosing to stop them deletes every active reminder; unarchiving does not bring them back.
+1 -1
View File
@@ -44,7 +44,7 @@ Schedule 将一次性、固定周期、按每日、按每周以及 cron 本地
cron 输入携带 `expression` 和 `time_zone`。表达式是标准五字段 Vixie 形式 `minute hour day-of-month month day-of-week`:minute 0-59、hour 0-23、day-of-month 1-31、month 1-12、day-of-week 0-7,其中 `0` 和 `7` 都表示周日。每个字段接受 `*`、单个值、`a-b` 范围、`n >= 1` 的 `*/n` 与 `a-b/n` 步长,以及由这些形式组成的逗号分隔列表。`L`、`W`、`#`、`JAN`/`MON` 名称、`@daily` 风格宏、六字段表达式、超出范围的值、反向范围、零步长和空字段都会以 `invalid_rule` 拒绝,且错误信息会指出违规字段。由于该方言只有五个字段,最小间隔为一分钟。创建时存储规范化表达式:重复值合并,相邻的值与范围合并,均匀步长写为 `a-b/n`,而以 `*` 开头的字段写为星号步长(匹配全部取值时写 `*`,否则写最宽的星号步长并追加其余取值),步长 `1` 被移除,周日统一写为 `0`。不以 `*` 开头的字段绝不会变成星号步长,因此下文的日规则在存储后保持不变。持久化解码器拒绝非规范化的已存储表达式,因此记录始终保存规范化文本。day-of-month 或 day-of-week 中任一为星号字段时,本地日期需两个字段都匹配;两者都不是星号字段时,匹配任一字段即可。字段文本以 `*` 开头即为星号字段,与它匹配的取值集合无关,因此步长星号字段与其他字段同时约束,而不是替代它。首个目标是该时区内本地日期与时间匹配的第一个严格未来时点,使用与 Daily 相同的缺口跳过与重叠取较早时点规则。完整方言与规范化规则见 [cron 时间语义](../../../docs/subsystems/schedule.zh.md#cron-wall-clock-input)。
提醒绑定到调用 Agent 的会话。共享的 `schedule` Remote namespace 提供 `catalog`,返回活动和未运行的宿主任务及其原始会话 id,并提供带明确会话 id 的 `list`、`history`、`update` 和 `delete`。它还提供 `deleteInactive`,删除所有会话中全部未运行(界面显示为「已结束」)任务并返回移除的行数;未运行任务从不会安排定时器,因此该删除不重算任何目标,且该批删除是一次持久写入,失败时不删除任何行,调用以拒绝结束而不返回计数。Remote `list` 和模型 `schedule_list` 仅返回活动任务。目录条目的 `lastDelivery` 仅保留最近一次回执;目录和列表响应均不包含已保存的发送历史。这些操作均不激活 Agent 或读取会话日志。显式删除会阻止后续投递,保留原会话和已经入队的消息,并移除已存储的任务行及其已保存的发送记录:任务从 `list` 和 `catalog` 中消失、不再调度,`history` 对相同的 `(sessionId, id)` 返回 `schedule_not_found`。
提醒绑定到调用 Agent 的会话。共享的 `schedule` Remote namespace 提供 `catalog`,返回活动和未运行的宿主任务及其原始会话 id,并提供带明确会话 id 的 `list`、`history`、`update` 和 `delete`。它还提供 `deleteInactive`,删除所有会话中全部未运行(界面显示为「已结束」)任务并返回移除的行数;未运行任务从不会安排定时器,因此该删除不重算任何目标,且该批删除是一次持久写入,普通失败时不删除任何行,调用以拒绝结束而不返回计数;后端报告 `unknown-state` 时可能已移除部分行,服务会先发出 `schedule/changed` 再拒绝,客户端重新读取。Remote `list` 和模型 `schedule_list` 仅返回活动任务。目录条目的 `lastDelivery` 仅保留最近一次回执;目录和列表响应均不包含已保存的发送历史。这些操作均不激活 Agent 或读取会话日志。显式删除会阻止后续投递,保留原会话和已经入队的消息,并移除已存储的任务行及其已保存的发送记录:任务从 `list` 和 `catalog` 中消失、不再调度,`history` 对相同的 `(sessionId, id)` 返回 `schedule_not_found`。
归档仍有活动提醒的会话会被拒绝,直到这些提醒停止;选择停止它们会删除全部活动提醒,取消归档不会把它们带回来。
+6
View File
@@ -374,6 +374,8 @@ export class ScheduleService extends TypertRemoteService {
* slot, so a delivery ordered behind this call ends its task after the
* enumeration rather than during it. The batch is one durable write: a failure
* removes nothing and rejects, so the caller never receives a partial removal.
* A backend that reports `unknown-state` may have removed rows, so the service
* announces the change before rejecting and clients re-read the catalog.
* @returns Count of removed rows; zero when the Host held no ended task.
*/
@Remote('deleteInactive')
@@ -504,6 +506,10 @@ export class ScheduleService extends TypertRemoteService {
try {
removed = await (await this.getDomain()).table('tasks').deleteMany(ids)
} catch (error: unknown) {
// A rejection never establishes which rows landed, and the backend can
// report `unknown-state` after removing some of them. Announce the change
// so observers re-read instead of trusting their stale catalog.
this.emitChanged()
throw error instanceof Error ? error : new Error(`${label}: ${String(error)}`)
}
// Rows enumerated in this slot are present, so a non-empty batch always removed rows.
@@ -5,6 +5,7 @@ import { afterEach, beforeEach, expect, it, vi } from 'vitest'
import { createAfterScheduleRecord, createEveryScheduleRecord, ScheduleId } from '../src/domain.ts'
import type { DomainFacility } from '@deepseek-ai/dsh-storage-domain'
import { scheduleDomain, type ScheduleTask } from '../src/storage.ts'
import { StorageError } from '@deepseek-ai/dsh-storage'
import { MemoryMediaPool } from '../../../storage/storage-domain/tests/helpers/memory-backend.ts'
import { gatedScheduleBackend, harness } from './harness.ts'
@@ -195,13 +196,52 @@ it('removes nothing and reports the batch failure', async () => {
expect(failure).toBeInstanceOf(Error)
expect(String(failure)).toContain('injected batch failure')
// One write carried both rows; the rejection left both stored and announced nothing.
// One write carried both rows; the rejection may have landed some of them, so
// observers are told to re-read even though both rows remain in this catalog.
expect(batches).toEqual([['ended-a', 'ended-b']])
expect(changed).not.toHaveBeenCalled()
expect(changed).toHaveBeenCalledTimes(1)
expect((await service.catalog()).map(entry => entry.title).sort())
.toEqual(['Partial first', 'Partial second'])
})
it('quarantines the Host domain when the backend reports unknown-state', async () => {
const pool = new MemoryMediaPool()
let batches = 0
const backend = gatedScheduleBackend(pool, {
deleteRecords: async (table) => {
if (table !== 'tasks') return
batches += 1
throw new StorageError('unknown-state', 'injected unknown-state')
},
})
const test = await harness({
pool,
backend,
beforeService: async (_ctx, facility) => {
const domain = await facility.open(scheduleDomain)
await domain.table('tasks').put(ScheduleId('ended-a'), endedRow(SessionId('quarantine-a'), 'ended-a', 'Quarantine first'))
await domain.table('tasks').put(ScheduleId('ended-b'), endedRow(SessionId('quarantine-b'), 'ended-b', 'Quarantine second'))
await domain.close()
},
})
tests.push(test)
const { ctx, service } = test
const changed = vi.fn()
ctx.on('schedule/changed', changed)
const failure = await service.deleteInactive().then(() => undefined, (error: unknown) => error)
expect(failure).toMatchObject({ name: 'StorageError', code: 'unknown-state' })
expect(batches).toBe(1)
expect(changed).toHaveBeenCalledTimes(1)
// The quarantined domain rejects reads instead of serving stale rows.
await expect(service.catalog()).rejects.toMatchObject({ code: 'unknown-state' })
// Later batches are rejected before reaching the backend again.
await expect(service.deleteInactive()).rejects.toMatchObject({ code: 'unknown-state' })
expect(batches).toBe(1)
// The durable rows are still there; only the in-process domain is quarantined.
expect(pool.media.get('schedule')?.tables.get('tasks')?.size).toBe(2)
})
it('reports a rejection reason that carries no Error as a labeled Error', async () => {
const pool = new MemoryMediaPool()
let batches = 0
@@ -233,7 +273,7 @@ it('reports a rejection reason that carries no Error as a labeled Error', async
expect(failure).toBeInstanceOf(Error)
expect(String(failure)).toContain('schedule ended-task removal failed')
expect(batches).toBe(1)
expect(changed).not.toHaveBeenCalled()
expect(changed).toHaveBeenCalledTimes(1)
expect((await service.catalog()).map(entry => entry.title).sort())
.toEqual(['Unreportable first', 'Unreportable second'])
})
@@ -195,7 +195,9 @@ describe('shared Schedule management', () => {
.then(() => undefined, (error: unknown) => error)
expect(failure).toBeInstanceOf(AggregateError)
expect(batches).toBe(1)
expect(changed).not.toHaveBeenCalled()
// The batch may have landed some rows under unknown-state, so the service
// announces the change before the rejection reaches the caller.
expect(changed).toHaveBeenCalledTimes(1)
expect((await service.catalog()).map(entry => entry.title).sort()).toEqual(['First', 'Second'])
})
@@ -222,8 +224,9 @@ describe('shared Schedule management', () => {
// Nothing landed, and a reason that carries no Error must not read as success
// with the reminder still armed.
expect(failure).toBeInstanceOf(AggregateError)
expect((failure as AggregateError).errors.map(String).some(text => text.includes('schedule stop failed'))).toBe(true)
expect(batches).toBe(1)
expect(changed).not.toHaveBeenCalled()
expect(changed).toHaveBeenCalledTimes(1)
expect((await service.catalog()).map(entry => entry.title)).toEqual(['First'])
})
@@ -4,8 +4,9 @@
* from memory; every write queues on the chain, awaits backend durability
* FIRST, then mutates memory, then emits `domain/changed` — a rejected
* backend write leaves memory untouched (no divergence between reads and the
* medium), and events carry values that equal the in-memory state at
* emission, in write order.
* medium) unless the backend reports `unknown-state`, which quarantines the
* domain: reads and later writes reject until close. Events carry values that
* equal the in-memory state at emission, in write order.
* @module @deepseek-ai/dsh-storage-domain/src/domain
*/
@@ -140,7 +141,7 @@ interface TableHost {
readonly unit: KvUnit
/** Queue one job on the domain's single write chain. */
enqueue<T>(job: () => Promise<T>): Promise<T>
/** Throw `closed` once the domain has fully closed (reads stay valid while draining). */
/** Throw `closed` after teardown, or the recorded `unknown-state` while quarantined. */
assertReadable(): void
/** Emit `domain/changed` for one durably landed write. */
emitChanged(change: DomainChanged): void
@@ -299,10 +300,11 @@ export class DomainImpl {
}
private assertReadable(): void {
if (this.failure !== undefined) throw this.failure
// Close wins: the KvUnit contract reports `closed` for calls after teardown.
if (this.closed) {
throw new DomainError('closed', `domain '${this.name}' is closed`)
}
if (this.failure !== undefined) throw this.failure
}
}
@@ -309,6 +309,43 @@ describe('KvTable writes', () => {
expect(observed).toEqual([[], []])
})
it('calls the backend batch primitive exactly once per deleteMany', async () => {
const inner = new MemoryStorageBackend()
let batches = 0
let singles = 0
const counting: StorageBackend = {
close: () => inner.close(),
kv: {
open: async (descriptor) => {
const kv = inner.kv
if (kv === undefined) throw new Error('memory backend is missing its kv facet')
const unit = await kv.open(descriptor)
return {
loadAll: () => unit.loadAll(),
putRecord: (table, key, value) => unit.putRecord(table, key, value),
deleteRecord: async (table, key) => {
singles += 1
return unit.deleteRecord(table, key)
},
deleteRecords: async (table, keys) => {
batches += 1
return unit.deleteRecords(table, keys)
},
setGlobal: value => unit.setGlobal(value),
close: () => unit.close(),
}
},
},
}
const { facility } = await harness({ backend: counting })
const table = (await facility.open(spec)).table('items')
await table.put('a', { label: 'x', count: 1 })
await table.put('b', { label: 'y', count: 2 })
await expect(table.deleteMany(['a', 'b', 'missing'])).resolves.toBe(2)
expect(batches).toBe(1)
expect(singles).toBe(0)
})
it('emits domain/changed per durable write, in order, with tombstones and global marker', async () => {
const { facility, changes } = await harness()
const domain = await facility.open(spec)
@@ -368,6 +405,10 @@ describe('durability failure', () => {
])
expect(pool.media.get('demo')!.tables.get('items')!.get('b')).toEqual({ label: 'y', count: 2 })
expect(changes).toHaveLength(seen)
// The write chain keeps serving after a rejected batch.
await table.put('c', { label: 'z', count: 3 })
expect(table.get('c')).toEqual({ label: 'z', count: 3 })
})
it('quarantines the domain after a batch reports unknown-state', async () => {
@@ -395,13 +436,17 @@ describe('durability failure', () => {
const table = domain.table('items')
await table.put('a', { label: 'x', count: 1 })
await expect(table.deleteMany(['a'])).rejects.toMatchObject({ code: 'unknown-state' })
// A later job queued before the failure surfaces is rejected too.
const failing = table.deleteMany(['a'])
const queued = table.put('b', { label: 'y', count: 2 })
await expect(failing).rejects.toMatchObject({ code: 'unknown-state' })
await expect(queued).rejects.toMatchObject({ code: 'unknown-state' })
// Synchronous reads and later chain jobs see the quarantined domain.
expect(() => table.get('a')).toThrowError(/unknown-state/)
expect(() => [...table.keys()]).toThrowError(/unknown-state/)
expect(() => table.size).toThrowError(/unknown-state/)
await expect(table.deleteMany(['missing'])).rejects.toMatchObject({ code: 'unknown-state' })
await expect(table.put('b', { label: 'y', count: 2 })).rejects.toMatchObject({ code: 'unknown-state' })
await expect(table.put('c', { label: 'z', count: 3 })).rejects.toMatchObject({ code: 'unknown-state' })
await expect(domain.global.set({ theme: 'dark' })).rejects.toMatchObject({ code: 'unknown-state' })
expect(() => domain.global.get()).toThrowError(/unknown-state/)
// Releasing the domain stays possible.
@@ -30,8 +30,8 @@
en: b7d222309b3e5438
zh: 807c7314ee341d94
/deepseek-ai-dsh-storage-json/understand-the-implementation/design-concept:
en: d4f84e0e5d2b3770
zh: 87eca52475ea2fe6
en: 7baa51f75f15a237
zh: 1415a2b73940e41d
/deepseek-ai-dsh-storage-json/understand-the-implementation/file-formats:
en: b3de0a697db4f916
zh: 56b26f7823392bd5
+1 -1
View File
@@ -70,7 +70,7 @@ The two layouts share atomic publication but assign state ownership differently.
### Design concept
- **`single` keeps memory authoritative.** Each write changes the in-memory unit, serializes its complete state, and atomically replaces `<unit>.json`. A failed publish restores the prior in-memory value. A batch delete is one such publish: the whole batch is applied or restored together.
- **`per-record` keeps the directory authoritative.** Each put or delete changes one `<unit>/<table>/<key>.json` document, and `loadAll()` rereads the tree. A batch delete renames the batch's documents out of the readable set, restores them if a rename fails, and unlinks the staged copies afterwards. If that restoration fails, the unit rejects with `unknown-state` and every later operation on it rejects. Each document stamps the unit version and carries one record value.
- **`per-record` keeps the directory authoritative.** Each put or delete changes one `<unit>/<table>/<key>.json` document, and `loadAll()` rereads the tree. A batch delete renames the batch's documents out of the readable set, restores them if a rename fails, and unlinks the staged copies afterwards. If that restoration fails, the unit rejects with `unknown-state` and every later operation on it rejects. The batch's renames and unlinks are not followed by a directory fsync, so crash durability follows the filesystem's entry-update guarantees: a crash can resurrect a staged document or leave `.deleting` residue. Each document stamps the unit version and carries one record value.
- **Publication is durable per call.** A write uses a temporary file, fsync, atomic `rename()` replacement, and a parent-directory fsync on POSIX. The domain layer's write chain supplies ordering across calls.
### File formats
+1 -1
View File
@@ -70,7 +70,7 @@ kind: "package-reference"
### 设计理念
- **`single` 以内存为权威状态。** 每次写入都会更改内存单元、序列化其完整状态,并以原子方式替换 `<unit>.json`。发布失败会恢复先前的内存值。批量删除就是这样一次发布:整批一起应用或一起恢复。
- **`per-record` 以目录为权威状态。** 每次 put 或 delete 都会更改一个 `<unit>/<table>/<key>.json` 文档,`loadAll()` 则重新读取目录树。每份文档都带有单元版本戳与一条记录值。批量删除先把整批文档改名移出可读集合,任一次改名失败就恢复,之后删除暂存副本;恢复本身失败时该 unit 以 `unknown-state` 拒绝,之后对它的所有操作都会拒绝。
- **`per-record` 以目录为权威状态。** 每次 put 或 delete 都会更改一个 `<unit>/<table>/<key>.json` 文档,`loadAll()` 则重新读取目录树。每份文档都带有单元版本戳与一条记录值。批量删除先把整批文档改名移出可读集合,任一次改名失败就恢复,之后删除暂存副本;恢复本身失败时该 unit 以 `unknown-state` 拒绝,之后对它的所有操作都会拒绝。批量删除的改名与删除后不做目录 fsync,因此崩溃后的持久性取决于文件系统的目录项更新保证:崩溃可能让暂存文档重新出现,或留下 `.deleting` 残留。
- **每次调用都持久发布。** 写入过程使用临时文件、fsync、原子 `rename()` 替换,并在 POSIX 上 fsync 父目录。领域层写入链负责安排跨调用的顺序。
### 文件格式
@@ -24,7 +24,7 @@ vi.mock('node:fs/promises', async (importOriginal) => {
}
})
const DESCRIPTOR = { name: 'batch', version: 1, layout: 'per-record' as const, tables: ['t'], hasGlobal: false }
const DESCRIPTOR = { name: 'batch', version: 1, layout: 'per-record' as const, tables: ['t'], hasGlobal: true }
const roots: string[] = []
async function freshRoot(): Promise<string> {
@@ -49,6 +49,8 @@ it('waits for the whole batch before close releases the unit', async () => {
const unit = await backend.kv.open(DESCRIPTOR)
await unit.putRecord('t', 'a', { v: 1 })
await unit.putRecord('t', 'b', { v: 2 })
// An empty batch is a settled no-op that never touches the medium.
await expect(unit.deleteRecords('t', [])).resolves.toBeUndefined()
const firstEntered = Promise.withResolvers<undefined>()
const firstRelease = Promise.withResolvers<undefined>()
@@ -116,7 +118,14 @@ it('rejects with unknown-state when a failed batch cannot restore its staged doc
await expect(unit.loadAll()).rejects.toMatchObject({ code: 'unknown-state' })
await expect(unit.putRecord('t', 'c', { v: 3 })).rejects.toMatchObject({ code: 'unknown-state' })
await expect(unit.deleteRecord('t', 'a')).rejects.toMatchObject({ code: 'unknown-state' })
await expect(unit.setGlobal({ g: 1 })).rejects.toMatchObject({ code: 'unknown-state' })
await expect(unit.backupRecord('t', 'b')).rejects.toMatchObject({ code: 'unknown-state' })
// Releasing the handle stays possible.
await unit.close()
// A reopened unit reads the partial tree: the staged document is unreadable
// residue and the second record is still present.
const reopened = await backend.kv.open(DESCRIPTOR)
expect(await reopened.loadAll()).toEqual({ tables: { t: { b: { v: 2 } } }, global: null })
await reopened.close()
await backend.close()
})
@@ -30,8 +30,8 @@
en: 6867b52cbb94e572
zh: 187d5d61980c8fe7
/deepseek-ai-dsh-storage-sqlite/understand-the-implementation/design-concept:
en: 54ee8d9d7746e82d
zh: 510c8bca88d2495c
en: af6e4b3df6d321cd
zh: 9922901704a6780f
/deepseek-ai-dsh-storage-sqlite/understand-the-implementation/open-sequence:
en: 50d85e8d8e8b5bc1
zh: 2a7067bd7fb47b35
+1 -1
View File
@@ -69,7 +69,7 @@ The backend is a document-per-row layout over one `node:sqlite` connection, desi
### Design concept
- **Document per row.** Each unit table becomes a physical STRICT table `u_<unit>_<table> (key TEXT PRIMARY KEY, value TEXT)` whose `value` column holds the record's JSON text; the global singleton lives in a shared `unit_globals` table. One key update touches exactly one row — the reason to route a high-churn domain here.
- **Single-statement atomicity.** Every write primitive is one prepared statement, so SQLite's per-statement atomicity satisfies the KV contract without explicit transactions; write ordering stays the caller's responsibility (the domain layer's write chain).
- **Statement atomicity.** Every single-record write is one prepared statement; a batch delete runs its deletes in one explicit transaction, so the batch commits or rolls back together. A batch that cannot open or roll back its transaction rejects with `unknown-state`, after which every later operation on the unit rejects; write ordering stays the caller's responsibility (the domain layer's write chain).
- **Names validated before DDL.** Unit and table names must match `UNIT_NAME_RE` before they reach DDL, so no external input is ever interpolated into SQL identifiers.
- **Versions fail loud.** The physical layout version lives in `PRAGMA user_version` (fresh databases stamp it last); unit format versions live in the `units` table. Any other stamped value rejects — no migrations.
+1 -1
View File
@@ -69,7 +69,7 @@ kind: "package-reference"
### 设计理念
- **每行一份文档。** 每个单元表都变成一张物理 STRICT 表 `u_<unit>_<table> (key TEXT PRIMARY KEY, value TEXT)`,其 `value` 列保存记录的 JSON 文本;全局单例存放在共享的 `unit_globals` 表中。一个键的更新恰好触碰一行——这就是把高频变更领域路由到这里的原因。
- **语句原子性。** 每个单记录写入都是一条预处理语句;批量删除在一个显式事务中执行其删除,因此整批一起提交或回滚。写入顺序仍由调用方负责(领域层的写入链)。
- **语句原子性。** 每个单记录写入都是一条预处理语句;批量删除在一个显式事务中执行其删除,因此整批一起提交或回滚。无法开启或回滚事务的批量删除以 `unknown-state` 拒绝,之后对该 unit 的所有操作都会拒绝;写入顺序仍由调用方负责(领域层的写入链)。
- **名称在 DDL 之前校验。** 单元名与表名在进入 DDL 之前必须匹配 `UNIT_NAME_RE`,因此任何外部输入都不会被插值进 SQL 标识符。
- **版本明确报错。** 物理布局版本存放在 `PRAGMA user_version`(全新数据库最后盖戳);单元格式版本存放在 `units` 表中。任何其他已标记值都会被拒绝——不做迁移。
+21 -5
View File
@@ -30,6 +30,8 @@ export class SqliteKvUnit implements KvUnit {
private readonly globalUpsert: StatementSync | undefined
private readonly globalSelect: StatementSync | undefined
private closed = false
/** Set when a batch transaction could not be opened or rolled back. */
private unusable: StorageError | undefined
/**
* @param db - Open database handle owned by the backend (never closed here).
@@ -112,7 +114,18 @@ export class SqliteKvUnit implements KvUnit {
deleteRecords(table: string, keys: readonly string[]): Promise<void> {
return this.settle(() => {
const remove = this.statementsFor(table).remove
this.db.exec('BEGIN IMMEDIATE')
try {
this.db.exec('BEGIN IMMEDIATE')
} catch (error: unknown) {
// An unopened transaction can only mean the connection is not in the
// state this unit expects; later statements would run in an unknown one.
this.unusable = new StorageError(
'unknown-state',
`kv unit '${this.descriptor.name}' could not open a batch transaction; its readable state is unknown`,
{ cause: error },
)
throw this.unusable
}
try {
for (const key of new Set(keys)) remove.run(key)
this.db.exec('COMMIT')
@@ -120,10 +133,12 @@ export class SqliteKvUnit implements KvUnit {
try {
this.db.exec('ROLLBACK')
} catch (rollbackError: unknown) {
// A failed rollback leaves the connection's state to its own next
// statement; both failures are reported, neither is discarded.
/* v8 ignore next -- ROLLBACK fails only on an unusable connection; the next statement reports it. */
throw new AggregateError([error, rollbackError], `kv unit '${this.descriptor.name}' batch delete failed and could not roll back`)
this.unusable = new StorageError(
'unknown-state',
`kv unit '${this.descriptor.name}' batch delete failed and could not roll back; its readable state is unknown`,
{ cause: new AggregateError([error, rollbackError], 'batch delete failed and rollback failed') },
)
throw this.unusable
}
throw error
}
@@ -166,6 +181,7 @@ export class SqliteKvUnit implements KvUnit {
if (this.closed) {
throw new StorageError('closed', `kv unit '${this.descriptor.name}' is closed`)
}
if (this.unusable !== undefined) throw this.unusable
}
private statementsFor(table: string): TableStatements {
@@ -1,4 +1,4 @@
import { afterEach, describe, expect, it } from 'vitest'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { Context } from '@deepseek-ai/cordis'
import { chmod, mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
@@ -67,6 +67,56 @@ describe('sqlite backend specifics', () => {
await backend.close()
})
it('reports unknown-state when a failed batch cannot roll back', async () => {
const path = await freshDbPath()
const backend = backendAt(path)
const unit = await backend.kv.open(DESCRIPTOR)
await unit.putRecord('records', 'a', { n: 1 })
await unit.putRecord('records', 'b', { n: 2 })
const db = new DatabaseSync(path)
db.exec('CREATE TRIGGER block_b BEFORE DELETE ON "u_specimen_records"'
+ " WHEN old.key = 'b' BEGIN SELECT RAISE(ABORT, 'blocked'); END")
const original = DatabaseSync.prototype.exec
const execSpy = vi.spyOn(DatabaseSync.prototype, 'exec').mockImplementation(function (this: DatabaseSync, sql: string) {
if (sql === 'ROLLBACK') throw new Error('rollback denied')
return original.call(this, sql)
})
try {
await expect(unit.deleteRecords('records', ['a', 'b'])).rejects.toMatchObject({ code: 'unknown-state' })
// Later statements would run inside the still-open transaction.
await expect(unit.putRecord('records', 'c', { n: 3 })).rejects.toMatchObject({ code: 'unknown-state' })
await expect(unit.loadAll()).rejects.toMatchObject({ code: 'unknown-state' })
} finally {
execSpy.mockRestore()
db.close()
}
// The denied rollback leaves the unit's connection in a transaction; the
// backend close is what releases it, and this test's file is discarded by
// afterEach.
await expect(unit.close()).resolves.toBeUndefined()
await backend.close()
})
it('reports unknown-state when a batch cannot open its transaction', async () => {
const path = await freshDbPath()
const backend = backendAt(path)
const unit = await backend.kv.open(DESCRIPTOR)
await unit.putRecord('records', 'a', { n: 1 })
const original = DatabaseSync.prototype.exec
const execSpy = vi.spyOn(DatabaseSync.prototype, 'exec').mockImplementation(function (this: DatabaseSync, sql: string) {
if (sql.startsWith('BEGIN')) throw new Error('begin denied')
return original.call(this, sql)
})
try {
await expect(unit.deleteRecords('records', ['a'])).rejects.toMatchObject({ code: 'unknown-state' })
await expect(unit.deleteRecord('records', 'a')).rejects.toMatchObject({ code: 'unknown-state' })
} finally {
execSpy.mockRestore()
}
await expect(unit.close()).resolves.toBeUndefined()
await backend.close()
})
it('materializes STRICT record tables and stamps the schema version', async () => {
const path = await freshDbPath()
const backend = backendAt(path)
+11 -7
View File
@@ -112,13 +112,17 @@ export interface KvUnit {
/**
* Delete several records durably as one batch. Idempotent: missing keys are
* no-ops. Resolves once every listed record is absent from the readable
* state and durable; a rejection leaves the batch unapplied, so no listed
* record was removed. A backend that stages records and then cannot restore
* them rejects with `unknown-state` instead: the readable state is then
* unknown and every later operation on that unit rejects with the same
* error, while {@link close} still releases the unit. A backend whose medium
* can apply a whole batch in one operation (the `single` document, a row
* store) must do that instead of repeating {@link deleteRecord} per key.
* state and the backend accepted the batch; a rejection leaves the batch
* unapplied, so no listed record was removed. A backend that cannot
* establish or restore the batch's state rejects with `unknown-state`
* instead: the readable state is then unknown, every later operation on
* that unit rejects with the same error, and only {@link close} still
* releases it. On the `single` layout a failure after the atomic replacement
* (a directory-fsync error) can leave the new file on the medium while the
* caller sees a rejection; the unit keeps its memory consistent and the next
* publish rewrites the file. A backend whose medium can apply a whole batch
* in one operation (the `single` document, a row store) must do that instead
* of repeating {@link deleteRecord} per key.
* @param table - Declared table name.
* @param keys - Record keys; duplicates are harmless. In the `per-record`
* layout every key becomes a path segment and must match `[a-zA-Z0-9_-]+`