Repair workspace grouping that a container update erases
Sync upstream and build images / sync-and-build (push) Successful in 9m9s
Sync upstream and build images / sync-and-build (push) Successful in 9m9s
Updating the container recreates it, so anything outside the mounted /workspace and state volume is gone. dsh joins a session to its project when the session is created (upstream's history bootstrap runs once per state volume and never again), and the workspace registry stores an absolute canonical directory path — so a project directory under /home/node, where the workspace picker opens by default, loses its chats to "Ungrouped" permanently on the next update. Ship dsh-workspace-doctor.mjs and run it from the entrypoint before dsh starts: it reports every project directory that no longer resolves and every chat recoverable from one, and with DSH_WORKSPACE_DOCTOR=repair it recreates the missing directories and re-accounts those sessions in the workspace registry. Session logs are never read for writing and never modified, no project is invented, no chat is moved into a project it did not run in, and the registry is copied to workspace.json.bak-<stamp> before the atomic, 0600, fsynced write dsh's own JSON backend performs. The doctor refuses to write a registry version it does not know, is idempotent, and defaults to report-only in the image (warn) so nothing changes unasked; the Unraid template defaults to repair. CI: the doctor joins the recipe hash and the push path list, and the image smoke test asserts it is installed and runs against a fresh volume (exit 2, the tool itself failing, is the only outcome that fails a build).
This commit was merged in pull request #1.
This commit is contained in:
1 parent
14d4f2b255
commit
c41d7fb403
5 files changed
+540
-9
No files matched your search
@@ -6,9 +6,9 @@ name: Sync upstream and build images
|
||||
# commits, same dates as https://github.com/deepseek-ai/deepseek-harness/tags.
|
||||
# 2. For every upstream version that is also published on npm (from
|
||||
# MIN_VERSION up): build the image if the registry lacks it, or rebuild it
|
||||
# in place when our recipe (Dockerfile, entrypoint, wrapper, bind patch),
|
||||
# pnpm, or the node base image changed since it was built. One image tag
|
||||
# per version — <version>, nothing else.
|
||||
# in place when our recipe (Dockerfile, entrypoint, doctor, wrapper, bind
|
||||
# patch), pnpm, or the node base image changed since it was built. One image
|
||||
# tag per version — <version>, nothing else.
|
||||
# 3. Point `:latest` at the newest release-candidate version.
|
||||
# 4. Publish one Gitea release per built version on the mirrored upstream
|
||||
# tag, titled like upstream, carrying upstream's own notes plus a short
|
||||
@@ -35,6 +35,7 @@ on:
|
||||
paths:
|
||||
- Dockerfile
|
||||
- docker-entrypoint.sh
|
||||
- dsh-workspace-doctor.mjs
|
||||
- run-as-runtime-user.sh
|
||||
- web.cordis.patch.yml
|
||||
- .gitea/workflows/sync-upstream.yml
|
||||
@@ -100,7 +101,7 @@ jobs:
|
||||
npm view @deepseek-ai/dsh versions --json | grep -o '"[0-9][^"]*"' | tr -d '"' > /tmp/npm-versions
|
||||
docker pull -q "$BASE_IMAGE" > /dev/null
|
||||
BASE_DIGEST=$(docker inspect --format '{{index .RepoDigests 0}}' "$BASE_IMAGE")
|
||||
RECIPE_HASH=$(cat Dockerfile docker-entrypoint.sh run-as-runtime-user.sh web.cordis.patch.yml | sha256sum | cut -c1-12)
|
||||
RECIPE_HASH=$(cat Dockerfile docker-entrypoint.sh dsh-workspace-doctor.mjs run-as-runtime-user.sh web.cordis.patch.yml | sha256sum | cut -c1-12)
|
||||
WANT="${PNPM_LATEST}|${BASE_DIGEST}|${RECIPE_HASH}"
|
||||
echo "pnpm=$PNPM_LATEST base=$BASE_DIGEST recipe=$RECIPE_HASH rebuild='$REBUILD'"
|
||||
|
||||
@@ -206,6 +207,14 @@ jobs:
|
||||
[ "$(docker exec "$SMOKE" dsh --version)" = "$V" ]
|
||||
[ "$(docker exec "$SMOKE" pnpm --version)" = "$PNPM_VERSION" ]
|
||||
[ "$(docker exec "$SMOKE" awk '/^Uid:/{print $2}' /proc/1/status)" = "1000" ]
|
||||
# The workspace doctor must be installed, executable and able to read
|
||||
# a state volume. --help is the strict presence check; --check may
|
||||
# legitimately answer 1 (something to report) on a fresh volume, so
|
||||
# only 2 — the tool itself failed — is a build failure.
|
||||
docker exec "$SMOKE" dsh-workspace-doctor.mjs --help > /dev/null
|
||||
DOCTOR_RC=0
|
||||
docker exec "$SMOKE" dsh-workspace-doctor.mjs --check > /dev/null || DOCTOR_RC=$?
|
||||
[ "$DOCTOR_RC" -le 1 ]
|
||||
OK=""
|
||||
for i in $(seq 1 30); do
|
||||
if docker exec "$SMOKE" node -e '
|
||||
|
||||
+2
-1
@@ -34,7 +34,7 @@ RUN npm install -g "@deepseek-ai/dsh@${DSH_VERSION}" "pnpm@${PNPM_VERSION}" \
|
||||
&& npm cache clean --force
|
||||
|
||||
COPY web.cordis.patch.yml /opt/deepseek-harness/web.cordis.patch.yml
|
||||
COPY docker-entrypoint.sh run-as-runtime-user.sh /usr/local/bin/
|
||||
COPY docker-entrypoint.sh run-as-runtime-user.sh dsh-workspace-doctor.mjs /usr/local/bin/
|
||||
|
||||
# Replace npm's bin symlinks for dsh and the pnpm family with the
|
||||
# privilege-dropping wrapper, resolving each real target at build time (never
|
||||
@@ -52,6 +52,7 @@ RUN set -eux; \
|
||||
done; \
|
||||
rm /usr/local/bin/run-as-runtime-user.sh; \
|
||||
chmod 0755 /usr/local/bin/docker-entrypoint.sh; \
|
||||
chmod 0755 /usr/local/bin/dsh-workspace-doctor.mjs; \
|
||||
mkdir -p /workspace /home/node/.dsh; \
|
||||
chown node:node /workspace /home/node/.dsh; \
|
||||
dsh --version; \
|
||||
|
||||
@@ -120,6 +120,13 @@ Three things went wrong with the alternatives, and this repo is shaped by each:
|
||||
`duplicate loader entry id: <id>` and no hint which plugin. Before every start the
|
||||
entrypoint compares the profile's plugin layers against the installed version's own
|
||||
bundle patches and logs the offending plugin with its remove command.
|
||||
- **Workspace grouping is checked before every start.** A project directory outside the
|
||||
mounted `/workspace` and state volume does not survive a container update, and dsh joins
|
||||
a session to its project only at creation — so those chats collect under "Ungrouped" and
|
||||
stay there. The entrypoint reports every such project and every recoverable chat, and
|
||||
with `DSH_WORKSPACE_DOCTOR=repair` recreates the missing directories and re-accounts the
|
||||
orphaned sessions at startup. Session logs are never touched; the workspace registry is
|
||||
backed up first. See [Chats land in Ungrouped after an update](#chats-land-in-ungrouped-after-an-update).
|
||||
|
||||
#### Operations
|
||||
- **Upstream mirror.** The `master` branch and the `dsh-v*` tags track GitHub exactly,
|
||||
@@ -205,6 +212,8 @@ PLUGINS: manage them with dsh's own tooling, e.g.
|
||||
docker exec DeepSeek-Harness dsh plugin --profile web add|update|remove PACKAGE
|
||||
pnpm ships in the image and dsh re-execs as the runtime user, so this cannot leave root-owned files in appdata.
|
||||
|
||||
WORKSPACE GROUPING: create projects under the Workspace path, not under /home/node. /home/node is the container's own filesystem and the next update erases it - a project directory there loses its files, and its chats drop to "Ungrouped" permanently because dsh only joins a chat to a project when the chat is created. The entrypoint checks this before every start and, with the Workspace Doctor variable below set to repair (the default here), recreates the erased directories and re-accounts the orphaned chats so they return to their project. A backup of the workspace registry is written before it touches anything; session logs are never modified.
|
||||
|
||||
Your DeepSeek API key can be set below (stored in the container env) or entered later in Settings once running.
|
||||
</Overview>
|
||||
<Category>AI: Tools:</Category>
|
||||
@@ -220,6 +229,7 @@ Your DeepSeek API key can be set below (stored in the container env) or entered
|
||||
<Config Name="PUID" Target="PUID" Default="1000" Description="User id dsh runs as, and the owner applied to the appdata volume at startup. Set to 99 to keep Unraid's usual nobody ownership." Type="Variable" Display="advanced" Required="false" Mask="false">1000</Config>
|
||||
<Config Name="PGID" Target="PGID" Default="1000" Description="Group id dsh runs as. Set to 100 (users) alongside PUID=99 for Unraid's usual share ownership." Type="Variable" Display="advanced" Required="false" Mask="false">1000</Config>
|
||||
<Config Name="Take ownership of Workspace" Target="DSH_CHOWN_WORKSPACE" Default="0" Description="Set to 1 to recursively chown the Workspace contents to PUID:PGID at startup, so the agent can modify files that arrived owned by someone else. Off by default - it rewrites ownership across your data." Type="Variable" Display="advanced" Required="false" Mask="false">0</Config>
|
||||
<Config Name="Workspace Doctor" Target="DSH_WORKSPACE_DOCTOR" Default="repair" Description="Workspace grouping at startup. warn (default in the image) only reports; repair recreates project directories that the container update erased and re-accounts the chats orphaned with them, so they return to their project instead of 'Ungrouped' (a backup of the workspace registry is written first, session logs are never touched); off is silent. Project directories outside /workspace and appdata are erased by every update - keep project data in Workspace." Type="Variable" Display="advanced" Required="false" Mask="false">repair</Config>
|
||||
</Container>
|
||||
```
|
||||
|
||||
@@ -234,16 +244,16 @@ trusts.
|
||||
| Base | `node:22-bookworm-slim` + `git` + `ca-certificates` |
|
||||
| Installed | `@deepseek-ai/dsh@<version>` and `pnpm@<version>` as npm globals (exact versions recorded as image labels) |
|
||||
| Runs as | `PUID:PGID`, default **1000:1000**. The entrypoint starts as root to repair volume ownership, then `setpriv`s down — dsh itself is never root. |
|
||||
| `ENTRYPOINT` | `["docker-entrypoint.sh"]` — fixes ownership, warns about plugin layers that collide with the installed version, drops privileges, `exec`s `dsh "$@"` |
|
||||
| `ENTRYPOINT` | `["docker-entrypoint.sh"]` — fixes ownership, warns about plugin layers that collide with the installed version, runs the workspace doctor, drops privileges, `exec`s `dsh "$@"` |
|
||||
| `CMD` | `["web", "--patch", "/opt/deepseek-harness/web.cordis.patch.yml", "--no-open"]` |
|
||||
| Port | `3080` |
|
||||
| `/home/node/.dsh` | `DSH_HOME` — settings, credentials, sessions, plugins, pnpm store/cache/config. **Mount this.** Taken over at startup. |
|
||||
| `/workspace` | The agent's working directory (`WORKDIR`). Mount only what the agent may read and modify. Mount point made writable; contents untouched unless `DSH_CHOWN_WORKSPACE=1`. |
|
||||
| `/opt/deepseek-harness/web.cordis.patch.yml` | The baked bind patch: `webserver.host: 0.0.0.0`, `port: 3080` |
|
||||
| Env | `PUID`, `PGID`, `DSH_CHOWN_WORKSPACE`, plus `DSH_HOME` / `PNPM_HOME` / `XDG_*` pinned into the state volume |
|
||||
| Env | `PUID`, `PGID`, `DSH_CHOWN_WORKSPACE`, `DSH_WORKSPACE_DOCTOR` (`warn` default, `repair` fixes lost grouping, `off` is silent), plus `DSH_HOME` / `PNPM_HOME` / `XDG_*` pinned into the state volume |
|
||||
| Labels | `dev.milesward.dsh-version`, `.pnpm-version`, `.base-digest`, `.recipe-hash` — what CI's freshness check reads |
|
||||
|
||||
Three rules that are easy to get wrong:
|
||||
Four rules that are easy to get wrong:
|
||||
|
||||
- **`--patch` must come immediately after `web`.** It's a launcher flag; placed after the
|
||||
web command's own flags it fails with `unknown option '--patch'`.
|
||||
@@ -252,6 +262,10 @@ Three rules that are easy to get wrong:
|
||||
- **Don't pass `--user`.** Doing so blocks the ownership repair; the entrypoint then
|
||||
verifies the volume is writable and, if it isn't, exits with one line naming the fix
|
||||
instead of a Node stack trace. Use `PUID`/`PGID` to change ids instead.
|
||||
- **Keep projects under `/workspace`.** The workspace directory picker opens in
|
||||
`/home/node`, which the next container update erases: any project created there loses
|
||||
its directory and its chats drop to "Ungrouped" for good. See
|
||||
[Chats land in Ungrouped after an update](#chats-land-in-ungrouped-after-an-update).
|
||||
|
||||
## Versions and upgrades
|
||||
|
||||
@@ -294,6 +308,35 @@ that happens whenever you accept an update Unraid offers. Before you do:
|
||||
```
|
||||
4. **Apply the update** (or edit the Repository tag, if you pin).
|
||||
|
||||
### Chats land in Ungrouped after an update
|
||||
|
||||
dsh lists a chat under the project directory it was created in, and it joins that project
|
||||
**when the session is created** — there is no later re-scan. (Upstream's history bootstrap
|
||||
runs once per state volume, on the first project, and never again.) Two consequences:
|
||||
|
||||
- A **project directory outside `/workspace` and `/home/node/.dsh`** — anything created
|
||||
under `/home/node`, where the workspace directory picker opens by default — lives in the
|
||||
container's own filesystem. Updating the container recreates it, that directory is gone,
|
||||
and its chats no longer match a project.
|
||||
- Adding the folder back creates a **fresh, empty** project; dsh never moves an existing
|
||||
chat into it. The old chats stay under "Ungrouped".
|
||||
|
||||
Before every start the entrypoint reports this — each project with its storage, each
|
||||
missing directory, every chat it could recover — and with `DSH_WORKSPACE_DOCTOR=repair`
|
||||
it fixes it at startup:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Repairs | recreates missing workspace directories, and re-accounts chats whose recorded directory matches a project but which no project lists |
|
||||
| Never | touches a session log, deletes a session, invents a project, or moves a chat into a project it did not run in |
|
||||
| Before writing | copies the workspace registry to `workspace.json.bak-<timestamp>`; the write itself is atomic and in dsh's own format |
|
||||
| By hand | `docker exec -u 1000:1000 <container> dsh-workspace-doctor.mjs --check` (add `--all --dry-run` to preview a repair) |
|
||||
|
||||
The recreated directory is empty: this restores the *grouping*, not the files — those lived
|
||||
in the container and nothing here can bring them back. Keep project data under `/workspace`
|
||||
and the situation cannot recur. A repair run against a *running* dsh can be overwritten by
|
||||
it; let the entrypoint do it at startup instead.
|
||||
|
||||
### What changes in `0.1.2+`: launch-token auth
|
||||
|
||||
From `0.1.2` the Web UI is gated by a one-time launch token:
|
||||
@@ -405,7 +448,8 @@ the official release, upstream safety check intact, no tampering.
|
||||
| `dsh-v*` tags | Upstream's tags, mirrored verbatim; releases attach to them. |
|
||||
| `main` | This branch — the build. |
|
||||
| `Dockerfile` | The image. `ARG DSH_VERSION` is the default for local builds; CI passes every version explicitly. |
|
||||
| `docker-entrypoint.sh` | Ownership repair + privilege drop, then `exec dsh "$@"`. |
|
||||
| `docker-entrypoint.sh` | Ownership repair + privilege drop, workspace-doctor run, then `exec dsh "$@"`. |
|
||||
| `dsh-workspace-doctor.mjs` | Reports (and optionally repairs) workspace grouping lost to a container update: missing project directories, chats orphaned from their project. |
|
||||
| `run-as-runtime-user.sh` | Installed over `dsh`, `pnpm`, `pnpx`, `pn`, `pnx`; re-execs root invocations as the runtime user. |
|
||||
| `web.cordis.patch.yml` | The `0.0.0.0` bind patch, applied via `--patch`. |
|
||||
| `.gitea/workflows/` | `sync-upstream.yml` (mirror · build · freshness · releases), `validate-suspect-image.yml` (forensics). |
|
||||
|
||||
@@ -129,6 +129,28 @@ if [ "$(id -u)" = 0 ]; then
|
||||
log " may be unable to modify them. Set DSH_CHOWN_WORKSPACE=1 to take ownership."
|
||||
fi
|
||||
|
||||
# Workspace grouping doctor. dsh joins a session to its project when the
|
||||
# session is created, against the project directory's canonical path — and a
|
||||
# directory outside the mounted /workspace and state volume does not survive
|
||||
# a container update. Those chats then collect under "Ungrouped" for good,
|
||||
# because the one-time history bootstrap never runs again. Report it always;
|
||||
# recreate the directories and re-account the orphaned sessions when asked
|
||||
# (backup first, session logs never touched). See README:
|
||||
# "Chats land in Ungrouped after an update".
|
||||
case "${DSH_WORKSPACE_DOCTOR:-warn}" in
|
||||
0|off|no) ;;
|
||||
repair|all|1|yes)
|
||||
/usr/local/bin/dsh-workspace-doctor --all \
|
||||
|| log "workspace doctor: repair did not finish; starting dsh with the registry unchanged"
|
||||
# The doctor runs as root here; hand what it wrote back to the runtime user.
|
||||
chown "$PUID:$PGID" "$STATE/storages/workspace.json" "$STATE/storages/"workspace.json.bak-* 2>/dev/null || true
|
||||
;;
|
||||
*)
|
||||
/usr/local/bin/dsh-workspace-doctor --check \
|
||||
|| log "workspace doctor: see the report above - set DSH_WORKSPACE_DOCTOR=repair to fix it at startup"
|
||||
;;
|
||||
esac
|
||||
|
||||
export HOME="$HOME_DIR"
|
||||
exec setpriv --reuid "$PUID" --regid "$PGID" --init-groups dsh "$@"
|
||||
fi
|
||||
@@ -141,4 +163,6 @@ if [ ! -w "$STATE" ]; then
|
||||
log " chown -R $(id -u):$(id -g) <host path mounted at $STATE>"
|
||||
exit 1
|
||||
fi
|
||||
/usr/local/bin/dsh-workspace-doctor --check \
|
||||
|| log "workspace doctor: see the report above (running unprivileged, so it only reports)"
|
||||
exec dsh "$@"
|
||||
@@ -0,0 +1,453 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* dsh-workspace-doctor — report, and optionally repair, DSH workspace grouping
|
||||
* in a container deployment whose state volume outlives the container itself.
|
||||
*
|
||||
* Why this exists
|
||||
* ---------------
|
||||
* dsh joins a session to a project ("Workspace") when the session is created in
|
||||
* that project's directory, plus a one-time history bootstrap. The workspace
|
||||
* record stores an absolute, realpath-canonicalized directory path, and a
|
||||
* session only keeps its membership while that path still resolves. Updating a
|
||||
* container recreates it: everything outside the mounted /workspace and state
|
||||
* volume — in particular a project directory created under $HOME — is gone.
|
||||
* Those sessions then drop out of their project and the chats collect under
|
||||
* "Ungrouped", permanently, because the history bootstrap never runs again.
|
||||
*
|
||||
* What this tool does
|
||||
* -------------------
|
||||
* It reads the state volume only. It never touches a session log, never deletes
|
||||
* anything, and never invents a project:
|
||||
* --repair-dirs recreate missing workspace directories (mkdir -p), so the
|
||||
* sessions already listed under them stay listed
|
||||
* --reattach re-account sessions whose recorded cwd matches an existing
|
||||
* workspace path but which no workspace lists, so those chats
|
||||
* reappear under their project
|
||||
* Any write is preceded by a timestamped backup of the workspace registry and
|
||||
* is written atomically, in the same format dsh itself writes.
|
||||
*
|
||||
* Usage: dsh-workspace-doctor [--check|--repair-dirs|--reattach|--all]
|
||||
* [--dry-run] [--state DIR] [--owner UID:GID] [--json]
|
||||
*/
|
||||
|
||||
import { copyFile, chown, mkdir, open, readFile, readdir, rename, rm, stat } from "node:fs/promises";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { existsSync, realpathSync, readFileSync } from "node:fs";
|
||||
import { dirname, join, resolve, sep } from "node:path";
|
||||
import zlib from "node:zlib";
|
||||
|
||||
/** Workspace domain unit this tool knows how to write. */
|
||||
const KNOWN_UNIT = { name: "workspace", version: 2 };
|
||||
/** Directories dsh can count on surviving a container update. */
|
||||
const PERSISTENT_ROOTS = ["/workspace"];
|
||||
/** Sessions above this size are skipped rather than read whole into memory. */
|
||||
const MAX_SESSION_BYTES = 64 * 1024 * 1024;
|
||||
|
||||
const USAGE = `Usage: dsh-workspace-doctor [options]
|
||||
|
||||
--check report only (default); exits 1 when it finds something to fix
|
||||
--repair-dirs recreate missing workspace directories
|
||||
--reattach re-account sessions that match a workspace but are not listed
|
||||
--all both repairs
|
||||
--dry-run print what a repair would change, write nothing
|
||||
--state DIR dsh state volume (default: $DSH_HOME or /home/node/.dsh)
|
||||
--owner U:G chown files this tool writes (default: $PUID:$PGID when run as root)
|
||||
--json machine-readable report
|
||||
-h, --help this text
|
||||
`;
|
||||
|
||||
/** Parse argv; unknown flags are fatal so a typo never becomes a silent no-op. */
|
||||
function parseArgs(argv) {
|
||||
const opts = {
|
||||
mode: "check",
|
||||
dryRun: false,
|
||||
json: false,
|
||||
state: process.env.DSH_HOME ?? "/home/node/.dsh",
|
||||
owner: undefined
|
||||
};
|
||||
const owner = process.env.PUID !== undefined && process.env.PGID !== undefined
|
||||
? `${process.env.PUID}:${process.env.PGID}`
|
||||
: undefined;
|
||||
for (let i = 0; i < argv.length; i += 1) {
|
||||
switch (argv[i]) {
|
||||
case "--check": opts.mode = "check"; break;
|
||||
case "--repair-dirs": opts.mode = "repair-dirs"; break;
|
||||
case "--reattach": opts.mode = "reattach"; break;
|
||||
case "--all": opts.mode = "all"; break;
|
||||
case "--dry-run": opts.dryRun = true; break;
|
||||
case "--json": opts.json = true; break;
|
||||
case "--state": opts.state = argv[++i] ?? die("--state needs a directory"); break;
|
||||
case "--owner": opts.owner = argv[++i] ?? die("--owner needs UID:GID"); break;
|
||||
case "-h":
|
||||
case "--help": process.stdout.write(USAGE); process.exit(0); break;
|
||||
default: die(`unknown option '${argv[i]}'`);
|
||||
}
|
||||
}
|
||||
if (opts.owner === undefined && process.getuid?.() === 0) opts.owner = owner;
|
||||
return opts;
|
||||
}
|
||||
|
||||
function die(message) {
|
||||
process.stderr.write(`dsh-workspace-doctor: ${message}\n`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
/** Read JSON, answering undefined for a missing file and throwing on corruption. */
|
||||
async function readJson(path) {
|
||||
try {
|
||||
return JSON.parse(await readFile(path, "utf8"));
|
||||
} catch (error) {
|
||||
if (error.code === "ENOENT") return undefined;
|
||||
throw new Error(`cannot read ${path}: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/** realpath when the path resolves, the normalized spelling otherwise. */
|
||||
function canon(path) {
|
||||
try {
|
||||
return realpathSync(path);
|
||||
} catch {
|
||||
return resolve(path);
|
||||
}
|
||||
}
|
||||
|
||||
/** Whether a path lives on a mount the container update preserves. */
|
||||
function storageOf(path, state) {
|
||||
if (path === state || path.startsWith(state + sep)) return "state volume";
|
||||
if (PERSISTENT_ROOTS.some((root) => path === root || path.startsWith(root + sep))) return "workspace mount";
|
||||
return "container filesystem";
|
||||
}
|
||||
|
||||
/** Version rank of a session log file, so a migrated log wins over its source. */
|
||||
function logRank(name) {
|
||||
const match = /\.v(\d+)\.jsonl(\.zstd)?$/.exec(name);
|
||||
if (match !== null) return Number(match[1]);
|
||||
return /\.jsonl(\.zstd)?$/.test(name) ? -1 : Number.NEGATIVE_INFINITY;
|
||||
}
|
||||
|
||||
/** First line of a session log — the header — decompressed when needed. */
|
||||
function headerLine(file) {
|
||||
let text;
|
||||
if (file.endsWith(".zstd")) {
|
||||
if (typeof zlib.zstdDecompressSync !== "function") return undefined;
|
||||
const raw = readFileSync(file);
|
||||
try {
|
||||
text = zlib.zstdDecompressSync(raw).toString("utf8");
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
} else {
|
||||
text = readFileSync(file, "utf8");
|
||||
}
|
||||
const end = text.indexOf("\n");
|
||||
return (end === -1 ? text : text.slice(0, end)).trim();
|
||||
}
|
||||
|
||||
/** Project every stored session to its durable header. */
|
||||
async function readSessions(sessionsRoot, warnings) {
|
||||
const sessions = [];
|
||||
if (!existsSync(sessionsRoot)) return sessions;
|
||||
for (const bucket of await readdir(sessionsRoot)) {
|
||||
const bucketDir = join(sessionsRoot, bucket);
|
||||
let entries;
|
||||
try {
|
||||
entries = await readdir(bucketDir, { withFileTypes: true });
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
for (const entry of entries) {
|
||||
if (!entry.isDirectory()) continue;
|
||||
const dir = join(bucketDir, entry.name);
|
||||
const files = (await readdir(dir)).filter((name) => logRank(name) !== Number.NEGATIVE_INFINITY);
|
||||
if (files.length === 0) continue;
|
||||
const log = files.sort((left, right) => logRank(right) - logRank(left))[0];
|
||||
const file = join(dir, log);
|
||||
const size = (await stat(file)).size;
|
||||
if (size > MAX_SESSION_BYTES) {
|
||||
warnings.push(`session '${entry.name}': log is ${size} bytes; skipped`);
|
||||
continue;
|
||||
}
|
||||
const line = headerLine(file);
|
||||
if (line === undefined) {
|
||||
warnings.push(`session '${entry.name}': cannot read header from ${log}; skipped`);
|
||||
continue;
|
||||
}
|
||||
let header;
|
||||
try {
|
||||
header = JSON.parse(line);
|
||||
} catch {
|
||||
warnings.push(`session '${entry.name}': header is not JSON; skipped`);
|
||||
continue;
|
||||
}
|
||||
if (typeof header.cwd !== "string") {
|
||||
warnings.push(`session '${entry.name}': header carries no cwd; skipped`);
|
||||
continue;
|
||||
}
|
||||
sessions.push({
|
||||
id: typeof header.id === "string" ? header.id : entry.name,
|
||||
cwd: header.cwd,
|
||||
createdAt: typeof header.createdAt === "number" ? header.createdAt : undefined,
|
||||
origin: header.origin,
|
||||
storeDir: bucket
|
||||
});
|
||||
}
|
||||
}
|
||||
return sessions;
|
||||
}
|
||||
|
||||
/** Apply ownership to what this tool wrote, when it runs as root. */
|
||||
async function applyOwner(path, owner, warnings, recursive = false) {
|
||||
if (owner === undefined) return;
|
||||
const [uid, gid] = owner.split(":").map(Number);
|
||||
if (!Number.isInteger(uid) || !Number.isInteger(gid)) {
|
||||
warnings.push(`--owner '${owner}' is not UID:GID; leaving ownership untouched`);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await chown(path, uid, gid, recursive);
|
||||
} catch (error) {
|
||||
warnings.push(`cannot chown ${path}: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Durably replace the registry exactly the way dsh's own JSON storage backend
|
||||
* does: same-directory temp file, mode 0600, fsync, atomic rename, directory
|
||||
* fsync. A crash mid-write must never leave a half registry behind.
|
||||
*/
|
||||
async function writeAtomic(path, data) {
|
||||
const tmp = join(dirname(path), `.workspace-doctor-${randomUUID()}.tmp`);
|
||||
try {
|
||||
const handle = await open(tmp, "wx", 0o600);
|
||||
try {
|
||||
await handle.writeFile(data, "utf8");
|
||||
await handle.sync();
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
await rename(tmp, path);
|
||||
} catch (error) {
|
||||
await rm(tmp, { force: true });
|
||||
throw error;
|
||||
}
|
||||
try {
|
||||
const dir = await open(dirname(path), "r");
|
||||
try {
|
||||
await dir.sync();
|
||||
} finally {
|
||||
await dir.close();
|
||||
}
|
||||
} catch {
|
||||
// A directory fsync failure costs crash-durability only; the write stands.
|
||||
}
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const opts = parseArgs(process.argv.slice(2));
|
||||
const state = resolve(opts.state);
|
||||
const registryPath = join(state, "storages", "workspace.json");
|
||||
const warnings = [];
|
||||
|
||||
const doc = await readJson(registryPath);
|
||||
if (doc === undefined) {
|
||||
process.stdout.write(`dsh-workspace-doctor: no workspace registry at ${registryPath} — nothing to do\n`);
|
||||
return 0;
|
||||
}
|
||||
const unit = doc.unit ?? {};
|
||||
const records = doc.tables?.workspaces;
|
||||
if (typeof records !== "object" || records === null) die(`${registryPath} has no workspaces table`);
|
||||
const knowFormat = unit.name === KNOWN_UNIT.name && unit.version === KNOWN_UNIT.version;
|
||||
|
||||
// Membership as dsh stores it: one owner per session id.
|
||||
const accounted = new Map();
|
||||
for (const [workspaceId, record] of Object.entries(records)) {
|
||||
for (const sessionId of record.sessionIds ?? []) {
|
||||
if (accounted.has(sessionId)) warnings.push(`session '${sessionId}' is accounted by two workspaces; left untouched`);
|
||||
else accounted.set(sessionId, workspaceId);
|
||||
}
|
||||
}
|
||||
|
||||
const sessions = await readSessions(join(state, "sessions"), warnings);
|
||||
const subagents = sessions.filter((session) => session.origin === "subagent").length;
|
||||
|
||||
// Which sessions belong to a workspace by their own recorded cwd.
|
||||
const reattachable = new Map();
|
||||
for (const workspaceId of Object.keys(records)) reattachable.set(workspaceId, []);
|
||||
for (const session of sessions) {
|
||||
if (session.origin === "subagent") continue;
|
||||
if (accounted.has(session.id)) continue;
|
||||
const cwd = canon(session.cwd);
|
||||
for (const [workspaceId, record] of Object.entries(records)) {
|
||||
if (canon(record.path) !== cwd) continue;
|
||||
reattachable.get(workspaceId).push(session);
|
||||
break;
|
||||
}
|
||||
}
|
||||
for (const list of reattachable.values()) list.sort((left, right) => (right.createdAt ?? 0) - (left.createdAt ?? 0));
|
||||
|
||||
// Directory health per workspace.
|
||||
const rows = [];
|
||||
for (const [workspaceId, record] of Object.entries(records)) {
|
||||
let missing = false;
|
||||
try {
|
||||
missing = !(await stat(record.path)).isDirectory();
|
||||
} catch {
|
||||
missing = true;
|
||||
}
|
||||
const wait = reattachable.get(workspaceId) ?? [];
|
||||
rows.push({
|
||||
workspaceId,
|
||||
path: record.path,
|
||||
title: record.title,
|
||||
dir: missing ? "MISSING" : "ok",
|
||||
missing,
|
||||
storage: storageOf(record.path, state),
|
||||
members: (record.sessionIds ?? []).length,
|
||||
reattach: wait.length,
|
||||
sessions: wait
|
||||
});
|
||||
}
|
||||
const missingDirs = rows.filter((row) => row.missing);
|
||||
const recoverable = rows.reduce((total, row) => total + row.reattach, 0);
|
||||
const ephemeral = rows.filter((row) => row.storage === "container filesystem");
|
||||
const doDirs = opts.mode === "repair-dirs" || opts.mode === "all";
|
||||
const doReattach = opts.mode === "reattach" || opts.mode === "all";
|
||||
|
||||
const report = {
|
||||
state,
|
||||
registry: registryPath,
|
||||
format: `${unit.name ?? "?"} v${unit.version ?? "?"}`,
|
||||
workspaces: rows.length,
|
||||
sessions: sessions.length,
|
||||
subagents,
|
||||
missingDirs: missingDirs.map((row) => row.path),
|
||||
ephemeral: ephemeral.map((row) => row.path),
|
||||
reattachable: rows.flatMap((row) => row.sessions.map((session) => ({ workspace: row.path, sessionId: session.id, cwd: session.cwd, createdAt: session.createdAt }))),
|
||||
repairedDirs: [],
|
||||
reattached: [],
|
||||
dryRun: opts.dryRun,
|
||||
warnings
|
||||
};
|
||||
|
||||
if (opts.json !== true) {
|
||||
process.stdout.write(`dsh-workspace-doctor: state=${state} registry=${report.format}\n`);
|
||||
process.stdout.write(`${rows.length} workspace(s), ${sessions.length} stored session(s) (${subagents} subagent)\n\n`);
|
||||
process.stdout.write(`${"PROJECT".padEnd(30)} ${"DIR".padEnd(8)} ${"STORAGE".padEnd(17)} ${"SESSIONS".padEnd(9)} REATTACH\n`);
|
||||
for (const row of rows) {
|
||||
process.stdout.write(`${row.path.slice(0, 29).padEnd(30)} ${row.dir.padEnd(8)} ${row.storage.padEnd(17)} ${String(row.members).padEnd(9)} ${row.reattach === 0 ? "-" : String(row.reattach)}\n`);
|
||||
}
|
||||
}
|
||||
|
||||
// Repair 1: a workspace whose directory is gone cannot take new sessions, and
|
||||
// the sessions it lists lose their validity on its next write. Recreate it.
|
||||
const dirsChanged = [];
|
||||
let failed = 0;
|
||||
if (doDirs) {
|
||||
for (const row of missingDirs) {
|
||||
if (opts.dryRun) {
|
||||
dirsChanged.push(row.path);
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
await mkdir(row.path, { recursive: true });
|
||||
// Everything under a path that did not exist was just created by us, so
|
||||
// it is ours to hand to the runtime user (root-run entrypoint, --owner).
|
||||
await applyOwner(row.path, opts.owner, warnings, true);
|
||||
dirsChanged.push(row.path);
|
||||
} catch (error) {
|
||||
failed += 1;
|
||||
warnings.push(`cannot create ${row.path}: ${error.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Repair 2: sessions dsh recorded in a workspace directory but never joined to
|
||||
// it (created before the project existed, or after the directory was lost).
|
||||
const reattached = [];
|
||||
if (doReattach) {
|
||||
for (const row of rows) {
|
||||
if (row.sessions.length === 0) continue;
|
||||
const record = records[row.workspaceId];
|
||||
const additions = row.sessions.filter((session) => !(record.sessionIds ?? []).includes(session.id));
|
||||
if (additions.length === 0) continue;
|
||||
reattached.push({ path: row.path, sessionIds: additions.map((session) => session.id) });
|
||||
if (opts.dryRun) continue;
|
||||
record.sessionIds = [...(record.sessionIds ?? []), ...additions.map((session) => session.id)];
|
||||
record.updatedAt = new Date().toISOString();
|
||||
}
|
||||
}
|
||||
|
||||
const changed = dirsChanged.length > 0 || reattached.length > 0;
|
||||
if (changed && opts.dryRun !== true) {
|
||||
if (!knowFormat) die(`refusing to write ${registryPath}: unknown registry format ${report.format} (expected ${KNOWN_UNIT.name} v${KNOWN_UNIT.version})`);
|
||||
const stamp = new Date().toISOString().replace(/[-:]/g, "").replace(/\..+$/, "");
|
||||
const backup = `${registryPath}.bak-${stamp}`;
|
||||
await copyFile(registryPath, backup);
|
||||
await writeAtomic(registryPath, `${JSON.stringify(doc, null, 2)}\n`);
|
||||
await applyOwner(registryPath, opts.owner, warnings);
|
||||
await applyOwner(backup, opts.owner, warnings);
|
||||
report.backup = backup;
|
||||
}
|
||||
report.repairedDirs = dirsChanged;
|
||||
report.reattached = reattached;
|
||||
|
||||
// Exit contract: a report says whether there is anything to fix; a repair says
|
||||
// whether it managed to. An ephemeral-but-repaired workspace is not a failure,
|
||||
// or every container start would log one.
|
||||
const pending = missingDirs.length > 0 || recoverable > 0;
|
||||
const exitCode = opts.mode === "check" || opts.dryRun ? (pending ? 1 : 0) : failed > 0 ? 1 : 0;
|
||||
|
||||
if (opts.json === true) {
|
||||
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
|
||||
return exitCode;
|
||||
}
|
||||
|
||||
if (changed) {
|
||||
process.stdout.write(`\n${opts.dryRun ? "would repair" : "repaired"}:\n`);
|
||||
for (const path of dirsChanged) process.stdout.write(` created directory ${path}\n`);
|
||||
for (const entry of reattached) process.stdout.write(` re-accounted ${entry.sessionIds.length} session(s) under ${entry.path}\n`);
|
||||
if (opts.dryRun) process.stdout.write(" (dry run: nothing was written)\n");
|
||||
else if (report.backup !== undefined) process.stdout.write(` backup: ${report.backup}\n`);
|
||||
}
|
||||
|
||||
if (recoverable > 0 && !doReattach) {
|
||||
process.stdout.write(`\n${recoverable} session(s) are recorded in a workspace directory but no workspace lists them:\n`);
|
||||
for (const row of rows) {
|
||||
for (const session of row.sessions) {
|
||||
const when = session.createdAt === undefined ? "unknown" : new Date(session.createdAt).toISOString().slice(0, 16);
|
||||
process.stdout.write(` ${session.id} ${when} ${row.path}\n`);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (missingDirs.length > 0 && !doDirs) {
|
||||
process.stdout.write(`\n${missingDirs.length} workspace directory(ies) are missing:\n`);
|
||||
for (const row of missingDirs) process.stdout.write(` ${row.path}\n`);
|
||||
}
|
||||
|
||||
const attention = [];
|
||||
if (missingDirs.length > 0 && !doDirs) attention.push(`${missingDirs.length} missing workspace director(ies)`);
|
||||
if (recoverable > 0 && !doReattach) attention.push(`${recoverable} recoverable ungrouped session(s)`);
|
||||
if (ephemeral.length > 0) attention.push(`${ephemeral.length} workspace(s) outside the persisted mounts`);
|
||||
|
||||
if (attention.length > 0) {
|
||||
process.stdout.write(`\n${attention.join(", ")}.\n`);
|
||||
if (ephemeral.length > 0) {
|
||||
process.stdout.write("Workspace directories outside /workspace and the state volume do not survive an\n");
|
||||
process.stdout.write("update: only their names are recreated here, never their files. Keep project data\n");
|
||||
process.stdout.write("under /workspace.\n");
|
||||
}
|
||||
if (!changed && (missingDirs.length > 0 || recoverable > 0)) {
|
||||
process.stdout.write("Fix: dsh-workspace-doctor --all (container: set DSH_WORKSPACE_DOCTOR=repair)\n");
|
||||
}
|
||||
} else {
|
||||
process.stdout.write("\nGrouping is intact: every workspace directory resolves and no session is orphaned.\n");
|
||||
}
|
||||
for (const warning of warnings) process.stdout.write(`warning: ${warning}\n`);
|
||||
return exitCode;
|
||||
}
|
||||
|
||||
try {
|
||||
process.exitCode = await main();
|
||||
} catch (error) {
|
||||
die(error.message);
|
||||
}
|
||||
Reference in new issue
Block a user