deepseek-harness/packages/storage/storage-json

description, kind

description	kind
JSON storage backend for hosts and maintainers choosing, configuring, or debugging whole-unit and per-record files under a configured root.	package-reference

@deepseek-ai/dsh-storage-json

English | 中文

Summary

dsh-storage-json stores domain data as readable JSON under a configured root and registers as backend json. Its default single layout keeps one complete <unit>.json file per unit; its per-record layout keeps one version-stamped document per record. Both layouts publish each changed file atomically, while the domain layer orders calls. Choose it when operators need inspectable files and the selected layout fits the write volume; choose SQLite for larger, frequently updated units. The backend is host-side only and contributes no prompt, tool, or schema.

Table of Contents

• Use this package
• Understand the implementation
• Further Exploration
• Model Experience
• Known Limitations and Deferred Work
• Dev Note

---

Use this package

Use this package when a composition needs readable, editable JSON storage. Route the relevant domains to backend json; each domain specification selects the single or per-record layout.

When to choose it

Choose the default single layout for small units that benefit from one complete, pretty-printed file. Choose per-record for disposable records whose point writes should replace only one document; unsupported or unreadable documents read as absent. Choose SQLite for frequent point updates in large units. The domain API has no multi-record transaction operation on either backend.

Configuration

The only plugin field is root, which holds the unit files and directories. It is required because the backend does not fall back to process.cwd(). The backend creates the root with mode 0o700 on demand. A domain specification selects its layout; this plugin has no layout override.

- name: '@deepseek-ai/dsh-storage'
- name: '@deepseek-ai/dsh-storage-json'
  config:
    root: /var/lib/dsh/data
- name: '@deepseek-ai/dsh-storage-domain'
  config:
    backend: json

Field	Default	Meaning
root	required	Directory holding <unit>.json files and <unit>/ trees; created 0o700 on demand

The generated configuration catalog is the exhaustive source for every accepted field and its JSDoc.

Observable behavior

Without a legacy bootstrap, a missing single file or per-record directory opens as an empty unit and materializes on the first write. In single, malformed content rejects with malformed-medium, and a stored version outside the current and compatibleVersions set rejects with version-mismatch. An accepted older file stays unchanged on read; the next atomic write stamps the current version and preserves untouched values in declared tables. Deleting missing keys or an empty batch does not restamp. In per-record, each malformed or unreadable document, and each document whose version is outside the accepted set, reads as an absent record, so one bad document does not reject the unit. Version refusal does not delete the document. Per-record keys must match [a-zA-Z0-9_-]+; an unsafe key rejects before any file operation. Resolved writes are durable; per-record deletions sync their directory changes on POSIX before resolving. A batch interrupted before resolution can be partially applied. A batch that cannot restore its readable or durable state rejects with unknown-state, after which every later operation except close() rejects; calls after close reject with closed.

An unmaterialized per-record tree can initialize its declared tables from a valid <root>/<unit>.json whole-unit document only when the source unit name matches and its version is current or declared compatible. This bootstrap writes during loading: it leaves the source file unchanged and copies stored values into current-stamped documents without converting them. A source version outside the accepted set leaves the new tree empty. Any declared table directory, even when empty, or a declared global.json suppresses this initialization for the complete unit. Deleted records therefore remain absent while the retained source file stays unchanged.

A synchronous serialization failure restores the open single unit's previous memory and permits retry on that handle. An I/O failure can occur after file replacement, so a rejected write does not prove that disk data or its stamp stayed unchanged. Stop using and close the backend and its domains, then reopen to reload saved state before deciding whether to retry. See domain versioning for mixed records and downgrade behavior.

---

Understand the implementation

Implementation internals — click to expand

The two layouts share atomic publication but assign state ownership differently. single owns an in-memory unit projection; per-record treats its directory tree as authoritative.

Design concept

• single keeps memory authoritative. Each write changes the in-memory unit, serializes its complete state, and atomically replaces <unit>.json. A batch delete publishes once. Serialization failures restore every affected in-memory value; uncertain I/O outcomes require reopening instead of continuing from that snapshot.
• per-record keeps the directory authoritative. Each put or delete changes one <unit>/<table>/<key>.json document, and loadAll() rereads the tree. A batch delete renames the batch's documents out of the readable set, syncs the directory on POSIX, then unlinks the staged copies. A failed staging or sync restores the documents and syncs the restored directory. If that restoration or sync fails, the unit rejects with unknown-state and every later operation on it except close() rejects. Each document stamps the unit version and carries one record value.
• Publication is durable per call. A write uses a temporary file, fsync, atomic rename() replacement, and a parent-directory fsync on POSIX. The domain layer's write chain supplies ordering across calls.

File formats

A single document carries the unit identity, global singleton, and all tables:

{
  "unit": { "name": "workspace", "version": 1 },
  "global": null,
  "tables": { "workspaces": { "<key>": { "path": "/work/demo" } } }
}

A per-record table document at <root>/<unit>/<table>/<key>.json has the form { "version": 1, "record": <value> }; the optional global value uses <root>/<unit>/global.json. The format version comes from the domain specification.

Source map

File	Role
src/index.ts	Plugin entry: backend registration, root config, unit open/close table
src/single-unit.ts	One single unit: authoritative memory, write primitives, publish rollback
src/per-record-unit.ts	One per-record unit: tree reads, path-safe records, and one-document writes
src/format.ts	Whole-unit and record serialization with version validation
src/atomic.ts	Atomic file replacement: temp write, fsync, rename, directory fsync

---

Further Exploration

Read these pages when this backend's view is not enough: the subsystem reference is the authoritative contract, and the sibling backend shows the alternative medium.

• Storage subsystem — the backend contract, domain semantics, and generated API.
• Storage package map — the family's packages and their repository position.
• SQLite storage backend — the point-update medium for high-frequency data.
• domain KV storage Agent Note — the design behind the backend family and its deferred work.

---

Model Experience

Stored domain records

What the model sees

Nothing. This backend contributes no prompt, tool, or schema; it persists non-session domain data behind ctx.storage for host-side consumers only.

Token effect

Zero live-request tokens.

KV Cache effect

None — the backend never touches live request prefixes.

Known Limitations and Deferred Work

These limits define when this backend is a poor fit or needs special operational care. They are current package constraints, not a task backlog.

• single rewrites the whole unit — each write republishes the complete unit file; use per-record or route the domain to SQLite when this cost is too high.
• No cross-process write locking — two processes writing the same unit can interleave replacements; writes to the same file use last-completion wins.
• Windows rename without explicit write-through — durability relies on libuv's rename() (MoveFileExW with replacement); the stricter Win32 write-through publish helper from the session-log backend is planned to move down here when the log facet lands.
• No crash-atomic per-record batch — a crash before deletion resolves can leave a partial batch and .deleting residue. Completed deletions sync the staged directory changes on POSIX; cleanup residue is unreadable.

Dev Note

Working context for maintainers — click to expand

The Agent Note flags the whole-unit rewrite scale premise as a risk: if a second consumer lands on this backend at thousand-record scale before being routed to SQLite, rewrite cost surfaces earlier than expected. The mitigation is configuration — point routes at the SQLite backend — not a change to this package.