22 KiB
Storage
English | 中文
The storage subsystem persists everything that is not a session event log (session logs have their own seam — persistence.md). It is one optional capability, not part of the agent-loop spine, split as a capability seam: the hub and Service Definition (dsh-storage, ctx.storage), the Service Providers (dsh-storage-json, registered as json, and dsh-storage-sqlite, registered as sqlite), and the Consumer data form (dsh-storage-domain, ctx.storageDomain, also reachable as ctx.storage.domain) — the backend contract's only Consumer and the typed API everything else uses. The hub performs no IO itself: backends own media, data forms own semantics, and product packages never touch backends directly. Design record: domain KV storage Agent Note.
Source: packages/storage/storage/src/backend.ts · packages/storage/storage-domain/src/spec.ts · packages/storage/storage-domain/src/events.ts
The hub: ctx.storage
Storage (signatures) is a meeting point, not a store. ctx.storage.backend is a name → backend table: multiple backends stay mounted side by side, and which backend serves which consumer is that consumer's configuration (the domain layer's route table), never a hub-global choice. register(name, backend) returns the disposer; duplicate names and unknown lookups throw StorageError. Disposal only unregisters the name — the owning plugin closes the backend after unregistering. Each backend plugin also publishes a lifecycle-only service key (storageBackendServiceKey(name)), which form providers inject so their activation cannot race backend registration.
Data forms mount on the hub under a merge-extensible key map:
/**
* Data forms mountable on the hub, keyed by form name. Form owners extend
* this map via declaration merging (the domain layer merges
* `domain: DomainFacility`) and mount the facility in their `apply`.
*/
interface StorageForms {}
mount(form, facility) is an effect whose disposer unmounts; a second mount of the same key throws duplicate-mount. form(form) resolves a mounted facility and throws form-not-mounted until the owning plugin loads — assemblies order plugins accordingly rather than silently deferring. The domain layer merges domain: DomainFacility, so ctx.storage.domain and ctx.storageDomain are the same object.
The backend contract
/**
* One registered backend. A backend owns exactly one medium and shares its
* lifecycle across all facets; facets are optional members — a backend that
* cannot serve a data kind simply omits it, and resolution fails loud instead.
*/
interface StorageBackend {
/** Key-value operations; absent when this backend cannot serve them. */
readonly kv?: KvFacet
/**
* Drain in-flight writes across all open units and release the medium.
* Idempotent; concurrent and repeated calls resolve once teardown finishes.
* @returns resolution after the medium is released.
*/
close(): Promise<void>
}
A backend owns one medium (a file-tree root, a database file) and exposes optional operation groups; kv is the only shipped group. KvFacet.open(descriptor) opens one named unit — KvUnitDescriptor carries the name, current format version, optional compatible versions, table names, and whether a global singleton slot exists — and returns a KvUnit with loadAll, putRecord, deleteRecord, deleteRecords, setGlobal, and close. Unit and table names must match UNIT_NAME_RE (safe as a file name and as a SQL identifier segment); record keys are opaque strings in the single layout and match [a-zA-Z0-9_-]+ path segments in the per-record layout. A unit does not serialize concurrent writes — ordering belongs to the caller — whole-document and row-store calls are atomic on the medium, while a per-record batch interrupted before resolution can be partially applied. Resolved writes are durable; per-record deletions sync directory changes on POSIX. deleteRecords applies a whole batch in one call: it resolves once every listed record is absent from the readable state and the backend accepted the batch, and a rejection normally leaves the batch unapplied; a backend that cannot establish or restore the batch's state rejects with unknown-state, after which every later operation on that unit rejects with the same error until close() releases it (calls after close reject closed). A single-layout failure after the atomic replacement (a directory-fsync error) can leave the new file on the medium while the caller sees a rejection; close the backend and its domains, then reopen to inspect saved state before retrying. A single medium stamped outside the current and declared compatible versions rejects version-mismatch; a per-record document stamped outside the accepted set reads as absent. A medium that cannot be parsed as the unit rejects malformed-medium. backend.ts is the normative clause-by-clause contract, and the shared conformance suite in tests/contract.ts checks the shared clauses against each backend. The json backend republishes one whole human-readable file per unit atomically; the sqlite backend stores one document per row in one database for frequently updated data.
Declaring a domain
A domain is declared once by its owning package as a spec object — the single source of the domain's identity, layout, and record schemas (zod, so z.infer keeps consumer types un-duplicated):
/** Static declaration of one domain: identity, version, and record layout. */
interface DomainSpec {
/** Domain name; must match `UNIT_NAME_RE` (doubles as the backend unit name). */
readonly name: string
/** Current domain format version; reads enforce it according to the selected layout. */
readonly version: number
/**
* Document layout for the backend unit: `single` (the default) stores the
* whole unit as one document; `per-record` stores each record as its own
* document, for units whose records are large, sparse, or individually
* disposable — the projection cache — with per-record version checks.
* Row stores retain unit-wide versioning for either layout request.
*/
readonly layout?: 'single' | 'per-record'
/**
* Older domain versions whose stored records the current record schemas
* also accept (the declaring owner vouches for that, typically by
* declaring the fields older records lack as optional). Ordinary reads admit
* listed versions without restamping saved data; writes stamp {@link version}
* while leaving untouched records as saved. Current schemas must therefore
* accept mixed generations even under the current unit stamp.
*/
readonly compatibleVersions?: readonly number[]
/**
* What `open` does with a stored table record that fails its zod schema.
* Absent (the default), the whole open rejects with `invalid-record` —
* right for authoritative data. `'backup-and-skip'` is for domains whose
* records are disposable derived data: the backend moves the record's
* document aside (`KvUnit.backupRecord`), the failure is logged with
* its cause, and the open continues with the record absent. A backend
* without `backupRecord` (no per-record document to move) falls back
* to the rejecting default. The global slot always rejects.
*/
readonly invalidRecords?: 'backup-and-skip'
/** Optional global singleton slot. */
readonly global?: DomainGlobalSpec<unknown>
/** Table declarations keyed by table name; each name must match `UNIT_NAME_RE`. */
readonly tables: Record<string, DomainTableSpec>
}
defineDomain(spec) pins the spec's literal types and fails loud at the owner's module load, before any medium is touched: a domain or table name outside UNIT_NAME_RE, a version that is not a non-negative integer, or a global schema that accepts null all throw (null is the medium's "never written" sentinel, so a stored nullable global could not round-trip). domainTable<K, V>(schema) declares one table with a phantom compile-time key type (typically a branded id); descriptorOf(spec) projects the backend-facing unit descriptor.
Domain versioning
version names the current read/write format. compatibleVersions lists older versions whose records and global value the owner can decode with its current schemas; only nonnegative integers below version are allowed. Without that declaration, reads stay exact-version. Acceptance does not weaken record validation or permit a newer unknown version. Each row's saved fields must determine its meaning unambiguously; a unit stamp cannot distinguish different meanings of identical fields in mixed rows.
Ordinary compatible opens and reads do not restamp or normalize saved values. Whole-unit JSON publishes the current stamp and changed data in one atomic file replacement; SQLite commits its first updated unit stamp and mutation in one transaction; batch deletes always use one transaction. Deleting missing records or an empty batch does not advance the stamp. The stamp identifies an accepted format, not completion of value normalization; readers must accept mixed generations. A backend with a unit-wide stamp rejects an unsupported unit version.
Serialization failure before publication or successful SQLite rollback preserves the previous data and stamp. An I/O failure after JSON replacement, or a failure of SQLite rollback itself, can leave the saved outcome unknown despite rejection. Stop using and close the backend and its domains, then reopen to inspect saved state before deciding whether to retry. The backend READMEs define these failure cases.
JSON's per-record layout checks and stamps each document independently; an unsupported document reads as absent. Its existing whole-unit bootstrap is an exception to read-only loading: it copies accepted historical values into current-stamped documents without converting them. An existing declared table directory, even when empty, or a declared global document suppresses bootstrap. SQLite retains unit-wide version checks regardless of the layout request. See the JSON backend and SQLite backend for layout and recovery behavior.
For a compatible successor, bump version, keep all still-readable stamps in compatibleVersions, and extend the same schema's decoding/defaults. Consumers normalize values when rewriting selected records; Storage does not assign application record versions. Verify historical and skipped upgrades, mixed values, reads without writes, failure rollback, restart, other-unit isolation, and unknown-version refusal. Table removal and layout changes require their own retention and upgrade policy; a compatible stamp alone does not define either.
A v2 unit may contain untouched v1 values. A v3 reader accepting stamp 2 must still decode those values; accepting stamp 1 is also necessary for users who skipped v2 entirely. Removing old decoding requires rewriting and validating every affected record, including those already under a current unit stamp. An incompatible change needs an explicit, recoverable upgrade before opening with the new schema; the storage domain has no value-migration runner.
Keep a consistent backup before writing a new format if return to an older reader is required: that reader can refuse the new stamp after the first mutation. Lowering a saved stamp by hand does not reverse value conversion. Domain compatibility does not migrate SQLite's physical schema or JSON envelope formats, and changing a backend route does not transfer stored data. Session-format migrations and SQLite physical-layout versioning have separate owners and do not migrate domain values.
The open domain
/** One open domain, typed by its spec. */
interface Domain<S extends DomainSpec> {
/** Domain name from the spec. */
readonly name: string
/** Global singleton handle; a spec without `global` has no usable handle (`never`). */
readonly global: DomainGlobalHandleOf<S>
/**
* Resolve one declared table handle. Handles are stable — repeated calls
* return the same instance.
* @param name - Declared table name.
* @returns the typed table handle.
*/
table<N extends keyof S['tables'] & string>(name: N): KvTable<TableKeyOf<S, N>, TableValueOf<S, N>>
/**
* Close this domain: reject new writes immediately, drain already-queued
* writes (their events still emit), release the backend unit, then free
* the domain name for a later open. Idempotent — repeated calls share one
* teardown. The consumer owns this call (typically as its own `ctx.effect`
* disposer); the facility closes any domain left open when it unmounts.
* @returns resolution after the unit is released.
*/
close(): Promise<void>
}
Reads are synchronous from authoritative in-memory state: KvTable exposes get/entries/keys/size (snapshot iterators that stay stable while queued writes land), and the global handle's get() serves the spec's initial until the first set materializes the slot on the medium. Every write — put, delete, deleteMany, update, global.set — queues on one per-domain chain and reaches backend durability first, then mutates memory, then emits domain/changed; a rejected backend write does not publish a domain change; an unknown durable outcome requires reopening before further use of that snapshot; a backend that reports unknown-state instead quarantines the domain, and its reads and later writes reject until close() releases it. update(key, fn) is an atomic read-modify-write at its chain slot (a missing key rejects missing-key); delete of an absent key resolves false with no write and no event. deleteMany(keys) removes the present keys as one backend batch in one chain slot, resolves the removed count, and emits one domain/changed per removed key; a rejected backend write leaves every record in place and emits nothing; an unknown-state rejection quarantines the domain. Returned records are the stored objects themselves, not copies — replace via put/update, never mutate in place.
The domain facility: ctx.storageDomain
DomainFacility (signatures) opens declared domains over routed backends. Routing is the domain plugin's configuration, never the hub's: backend names the required default route and routes overrides it per domain name. open(spec) runs a strict sequence, each step failing the whole call: it rejects a name already open or still closing (already-open), resolves the route (backend-not-found), requires the backend's kv facet (facet-unsupported), opens the unit (backend version-mismatch/malformed-medium pass through), and validates every stored record and global against the spec's zod schemas (invalid-record with the offending table and key). The caller owns the returned handle and releases it with Domain.close(); domains still open when the plugin unmounts are closed by the facility, and a closed domain's name frees for reopening only after teardown fully completes. get(name) is an untyped diagnostic lookup onto the package-private DomainImpl runtime behind every typed handle; closeAll() is the unmount path.
The change event: domain/changed
Every durable write emits one event strictly after the backend acknowledged durability, in the domain's write-chain order (event entry):
/** Shared location fields of one durable domain change. */
interface DomainChangedBase {
/** Owning domain name. */
readonly domain: string
/** Table name; `''` for a global-singleton write. */
readonly table: string
/** Record key; `''` for a global-singleton write. */
readonly key: string
}
/** One durable domain change; a closed union — switch on `operation`. */
type DomainChanged = DomainChangedPut | DomainChangedDeleted
put (inserts, overwrites, and global writes) carries the new snapshot in value — never the old value; a diffing consumer keeps its own previous snapshot. deleted is a tombstone with no value. The event is a notification, not a transaction participant: the commit point has passed at emission, so a synchronously throwing listener is contained with a logged warning rather than rejecting the already-durable write, and emitted values equal the in-memory state at emission. The event is in-process only; cross-process change push is a recorded limitation (package README).
Cordis API
Generated from source by scripts/gen-cordis-catalog.ts (verified fresh by pnpm run verify-cordis-catalog in doc-sync; regenerate with pnpm run gen-cordis-catalog) — the language sides differ only in locale-specific paired document paths. Signature blocks use a ts cordis-catalog fence and keep the original source JSDoc; dispatch modes are defined in the primer, and the framework-inherited ctx API lives in cordis-api/inherited.md.
ctx.storage — Storage
The storage hub service. Backends register under backend; data forms mount under their StorageForms key and are reached as ctx.storage.<form>.
/**
* Mount a data-form facility on the hub. Mounting is an effect: the
* returned disposer unmounts the form.
* @param form - Form key declared in {@link StorageForms}.
* @param facility - The facility instance to expose.
* @returns the disposer that unmounts the form.
*/
mount<K extends keyof StorageForms>(form: K, facility: StorageForms[K]): () => void
/**
* Resolve a mounted data form.
* @param form - Form key declared in {@link StorageForms}.
* @returns the mounted facility.
*/
form<K extends keyof StorageForms>(form: K): StorageForms[K]
Source: packages/storage/storage/src/index.ts
ctx.storageDomain — DomainFacility
The mounted domain facility. Opens declared domains over routed backends; one facility instance owns the open-domain table and enforces single-open per domain name.
/**
* Open one declared domain. Steps, each failing the whole call: reject a
* name that is already open (`already-open`); resolve the backend route
* (`backend-not-found` passes through from the hub); require its `kv` facet
* (`facet-unsupported`); open the unit projected from the spec (backend
* `version-mismatch`/`malformed-medium` pass through); load and validate
* every stored record against the spec's zod schemas (`invalid-record`
* with the offending table and key — unless the spec declares
* `invalidRecords: 'backup-and-skip'` and the unit can move documents aside, in
* which case the failing record is backed up, logged, and skipped);
* construct the domain.
*
* Lifecycle: the CALLER owns the returned handle and closes it via
* `Domain.close()` (typically as its own `ctx.effect` disposer) — the
* facility does not tie the domain to any consumer fiber. Domains still
* open when the facility unmounts are closed by the plugin disposer.
* @param spec - The domain declaration, typically from `defineDomain`.
* @returns the opened domain handle, typed by the spec.
*/
async open<S extends DomainSpec>(spec: S): Promise<Domain<S>>
/**
* Look up an open domain by name, untyped. Diagnostic surface; typed
* consumers hold the handle returned by {@link open}.
* @param name - Domain name.
* @returns the open domain runtime, or `undefined` when not open.
*/
get(name: string): DomainImpl | undefined
/**
* Close every domain still open on this facility. The unmount path for
* consumers that never called `Domain.close()` themselves; closing is
* idempotent, so double-closing an already-closed domain is harmless.
* @returns resolution after every unit is released.
*/
async closeAll(): Promise<void>
Source: packages/storage/storage-domain/src/index.ts
domain/* events
domain/changed — emit
A domain record or the global singleton changed, emitted once per write strictly after the backend acknowledged durability. Events of one domain arrive in its write-chain order.
/**
* A domain record or the global singleton changed, emitted once per write
* strictly after the backend acknowledged durability. Events of one
* domain arrive in its write-chain order.
* @param change - domain, table (`''` for global), key (`''` for global),
* operation discriminant, and on `put` the new snapshot.
* @mode emit
*/
'domain/changed'(change: DomainChanged): void