deepseek-harness/.agents/notes/archived/architecture/2026-08-28-subprocess-native-containment.md

Agent Note: Native owners contain escaped subprocess descendants

Status: implemented Archived: 2026-09-30

English | 中文

Problem

Detached POSIX process groups, Windows direct-parent traversal, and PTY descendant scans describe only members that remain observable through one process relationship. A child can call setsid, reparent, or outlive its direct parent and leave those ranges, so terminating the apparent tree can return while work, ports, or files remain active. A direct target result also does not prove that every descendant has stopped.

The ordinary subprocess handle cannot solve this gap by publishing a PID. Linux scope setup and the Windows Job runner establish target identity asynchronously, a PID does not name the complete managed range, and consumers would be forced to infer whether startup committed. The public result, range ownership, private startup protocol, and packaged entry therefore need separate owners.

Decision

LocalSubprocessRuntime selects one provider-private managed-range owner before a target can execute. Eligible Linux ordinary and PTY launches enter a transient user-systemd scope; eligible Windows ordinary launches enter an unnamed kill-on-close Job owned by a private runner. Unsupported hosts use the existing weaker fallback with one provider-lifetime warning. The provider never replays a target after a selected native path may have executed it.

An ordinary SubprocessHandle has no PID or public startup state. .done reports the direct target result or startup/provider failure, terminate() signals the selected range, and waitForExit() succeeds only after that same range is proven empty. SubprocessTerminalHandle.pid remains part of the terminal contract because PTY identity and foreground inspection require it.

Linux scope and one-shot bootstrap

The first eligible Linux ordinary or PTY call in one runtime deeply checks the exact runner entry, the libc execve and fcntl bindings, the readable user manager, and literal-argv transient-scope support. Failed deep probes are retried, while the first success is cached. Each later eligible call still performs a lightweight user-manager reachability probe before target execution. Once selected, a scope, protocol, state-query, or pre-exec failure is reported through that launch and never switches to fallback.

The parent creates one 0700 directory with a complete 0600 launch-request.json containing the final target cwd and environment. The private DSH_SUBPROCESS_RUNNER value locates that request while the runner starts from the provider cwd and a bootstrap-safe environment. systemd-run --user --scope --quiet --collect --expand-environment=no registers its process in the scope, then the one-shot bootstrap removes and validates the request, changes to the target cwd, restores the complete target environment, resolves a bare executable with the target PATH rules, clears FD_CLOEXEC on fd 0 through fd 2, and calls libc execve() with the original argv. The bootstrap becomes the target in place and preserves its inherited stdio; it does not remain as a supervisor.

Request consumption or a manager observation of a loaded unit establishes scope ownership. Unit absence before either fact remains unresolved while the direct launcher is running. If that launcher exits while the request remains unconsumed, the direct result rejects with startup failure unless its observed signal matches a termination requested while the launcher was running. A matching signal preserves the actual exit outcome for both ordinary and PTY launches; a recorded startup error always takes precedence. Range observation independently resolves an empty range when the launcher has exited and the unit is absent. The parent checks this unresolved interval every 50 milliseconds; after establishment, state queries back off exponentially to the existing 5-second systemctl bound. Each query reads LoadState, ActiveState, and TasksCurrent: loaded inactive or failed, or an established unit becoming not-found/inactive or otherwise collected away, proves the range empty. active, activating, reloading, and deactivating remain nonterminal, except that once termination was requested and either the launcher has exited or the bootstrap has consumed its request, a still-active unit reporting no processes is an empty range. Request consumption proves the payload entered the scope, so zero members establishes quiescence even before Node receives the direct-process exit notification. Without request consumption, launcher exit remains necessary: the manager ends a scope only on the populated-to-empty transition, so a payload killed before it entered the cgroup never triggers one. That conclusion stops the leftover unit so transient units cannot accumulate, while an unreported or [not set] process count stays unknown and keeps waiting. Unknown or malformed combinations and unreadable manager results reject waitForExit() instead of claiming quiescence. terminate() wakes a sleeping observer for an immediate recheck, and settlement cancels the losing backoff sleep. A strict sibling startup-error.json carries only request/bootstrap or target pre-exec failure, and the parent removes this spawn's private paths at observable lifecycle completion.

The ordinary target result still comes from the same child process. The PTY path uses the same request and bootstrap without a resident runner, so the node-pty PID, process group, session leader, controlling terminal, foreground inputWaiting, /dev/tty, readiness, and direct terminal outcome retain their existing meanings while scope membership covers setsid and reparented descendants.

Windows runner and Job

The Windows parent starts the provider runner from a bootstrap cwd and environment, gives it the original target argv after a private -- delimiter, and waits for Node's runner spawn event before sending exactly one start request. A pre-spawn runner error preserves that direct launch failure while proving that no Job range existed, so the empty-range wait succeeds; post-spawn infrastructure failure remains uncertain and rejects range settlement. Node IPC otherwise carries idempotent terminate control and exactly one result. Runner fd 0 through fd 2 are isolated, fd 3 carries IPC, and fd 4 through fd 6 carry target stdin, stdout, and stderr. Fd 4 inherits the platform null-device descriptor when stdin is ignored and uses a pipe otherwise. The shared Win32 layer maps fds 4 through 6 to OS handles through Node's exported uv_get_osfhandle(), rejects null plus the unsigned UV_INVALID_OS_FILE_HANDLE and UV_INVALID_FILE_DESCRIPTOR sentinels exposed by Koffi, temporarily enables inheritance on valid handles, and passes them through STARTF_USESTDHANDLES. spawnCurrentTokenJobProcess requires a separately resolved applicationName and a complete target environment, which it sends as a sorted, double-NUL-terminated UTF-16LE block with CREATE_UNICODE_ENVIRONMENT, including =X: drive entries, without mutating the runner environment. After the suspended target enters the Job and resumes, the runner closes only fd 4 through fd 6; it never mutates or destroys Node's standard streams. The parent returns the pipe carriers as the ordinary handle's stdio, and user bytes never pass through IPC.

The runner is the sole owner of the target process handle and unnamed Job handle. spawnCurrentTokenJobProcess creates the target suspended, assigns it to a kill-on-close Job that disallows active breakaway, and resumes it only after assignment. The runner polls the direct process for the target exit code and the Job for active-process count. It exits successfully only after the direct result has been delivered through the IPC send callback and the Job has reported zero active processes; the parent maps only that clean exit to successful waitForExit().

The parent permanently latches a validated numeric target-exit as soon as it arrives, before the existing stdout/stderr close or bounded-drain barrier settles. .done waits only for that stdio barrier and then returns the latched result. A later Job query, range-settlement failure, IPC loss, or abnormal runner exit rejects only waitForExit() and cannot replace the direct result. Infrastructure failure before a valid target result rejects .done, but that rejection does not reveal whether target execution began. On disconnect or result-send failure, the runner stops protocol work, terminates and closes its only Job handle, and exits nonzero. Closing the last Job handle kills remaining members but does not convert the disconnected path into a successful quiescence proof.

Private dispatch and protocol

Source launches execute the package runner entry through the TypeScript source launcher, built launches resolve the @deepseek-ai/dsh-subprocess-local/runner export, and the Python SDK single-file executable enters through python/sdk-runtime/runtime-bootstrap.mjs. The Python runtime owns that bootstrap. It calls the public CLI export when the private selector is absent; otherwise it removes the selector and dispatches to the same subprocess runner core. The public dsh argument parser has no hidden runner mode, apps/cli builds only its public entry, and packaging ships no second Node executable.

The selector is a per-spawn locator or sentinel, not a credential or persistent format. Linux uses one strict request plus one optional strict startup-error file. Windows uses one IPC channel with closed start and terminate requests and exactly two result branches: target-exit with a numeric exitCode, and error with required name and message plus only optional code, syscall, and path; the parent derives signal: null. Pre-commit cancellation uses the same ordinary error record. The cancellation reason never crosses the wire, so a parent cancellation latch restores its first local reason exactly, including null or undefined. Missing, extra, mistyped, or unknown fields fail closed. Target environments may contain the selector name, including Windows case variants, because the provider transmits target state separately and restores it only after private selection is consumed.

Fallback and cleanup

Linux falls back before target execution when the exact bootstrap, modern readable user-systemd manager, or literal-argv scope is unavailable. Windows ordinary launch falls back when the runner entry, Win32 bindings, or current-token Job probe is unavailable. macOS ordinary launch, Windows ConPTY, and other unsupported hosts retain their existing PGID, taskkill /T, or identity-fenced PTY observation. The warning states that descendants escaping those observable relationships are not guaranteed to terminate or delay waitForExit().

Normal Cordis disposal starts direct-result and range observation independently, requests termination, and waits for every owned range. Consumer teardown does not inspect an ordinary PID; it retains the original operation or startup error while attempting terminate and final wait, and preserves cleanup failures in the consumer's existing error order. A confirmed empty range permanently disables later signalling against stale identities.

Native Windows fixtures retain the same Job-owned termination and final wait on success and failure. A descendant PID is observation-only: issuing taskkill after its exit can terminate an unrelated process that has reused the number, including another concurrent test worker.

During a JavaScript-observable host exit, LocalSubprocessRuntime synchronously force-terminates every still-live handle without promises or timers. Linux sends the existing direct fallback kill and the exact scope kill; Windows kills the runner so its only Job handle closes; PTY fallback scans remain best effort. Per-handle failures are contained and do not change the host's exit result. Termination modes in which JavaScript cannot run remain outside this listener's guarantee.

Existing decisions and supersession

This note owns the current native-containment mechanism. It partially updates the provider and no-PID facts in the subprocess seam, the Linux teardown facts in persistent PTY sessions, the native targets used by synchronous host-exit cleanup, the ordinary consumer of shared Win32 process primitives, and the private entry selected by the Python SDK profile runtime. Each note retains its other decision and remains active.

Verification

The host-exit disposal fixture compares the provider's listener identities before and after disposal and verifies that an independently registered exit listener remains installed. Total process listener counts cannot establish provider cleanup because other process-lifetime handlers may register after provider construction.

• Provider and Linux protocol suites pin synchronous NUL rejection before launch side effects, strict request/error decoding, target cwd and complete environment restoration, private-variable collision, symlink-sensitive PATH traversal with preserved argv, close-on-exec removal for inherited stdio, pre-exec error ownership, failed-deep-probe retry plus successful-deep-probe caching with per-call manager checks, the three scope-establishment states including requested versus unexpected exits with an unconsumed request, LoadState/ActiveState/TasksCurrent parsing, releasing a leftover scope left active with no processes beside the live-client, unterminated, and unset-count cases, reloading, terminate wake-up with losing-delay cancellation, bounded established-scope backoff, and exactly-once PTY managed-owner cleanup.
• Windows protocol and Win32 suites pin exactly two result branches, numeric-only target exits, ordinary-error start cancellation with raw parent-local reasons, the reduced name/message/code/syscall/path error record, the fixed 2/3/267 to ENOENT, 740 to EACCES, 5 to EPERM, 193 to EFTYPE, and remaining-code to UNKNOWN mapping, start delivery after runner spawn, empty-range settlement after pre-spawn failure, explicit ordinally sorted target environment blocks with =C: preservation and double-NUL termination, uv_get_osfhandle() carrier mapping and unsigned invalid-sentinel rejection, the null-device ignored-stdin carrier and piped non-ignored stdin, result-send and IPC-disconnect failures, direct-result latching before stdio settlement, active-process quiescence, and unique handle cleanup.
• A keyless bash-startup-timeout Session snapshot pins the model-facing timeout result. A Linux user-systemd fixture holds the launch request unconsumed at an input barrier and verifies cancellation plus range settlement.
• Real Linux user-systemd tests run one ordinary and one node-pty setsid/reparent scenario through the production entry. They prove scope signalling and collection, bare executable lookup, escaped-descendant termination, range settlement, and unchanged PTY PID, session, controlling-terminal, foreground-input, /dev/tty, readiness, and startup-failure semantics.
• Native Windows tests prove suspended creation, Job assignment before resume, inherited stdio, default descendant inheritance, direct result, termination, active-process zero, abnormal/disconnected runner cleanup, kill-on-close, and synchronous host-exit termination. Source, built, and Python packaged smokes enter the same runner core.
• Public seam types, local provider, LSP and subagent consumers, shell fixtures, READMEs, the Cordis catalog, and the keyless subprocess API snapshot contain no ordinary PID; terminal PID remains.

Alternatives considered

Keep PID, make it optional, or add a public started promise. Rejected because each representation exposes an asynchronous provider identity that does not name the managed range and invites consumers to infer startup or quiescence from the wrong fact.

Extend process-group, session, parent-tree, or PID scanning. Rejected because a process can leave each observed relationship; broader SID signalling can also reach unrelated processes when a PTY helper shares a launcher session. Native OS membership is persistent and independently queryable.

Let the parent own or reopen the Windows Job. Rejected because copied handles, named Jobs, OpenJobObject, process-handle handoff, and completion ports create multiple lifecycle owners without improving the direct-result contract. One runner can own target creation, Job membership, result production, and final handle closure.

Carry control or results through target stdio or files on Windows. Rejected because user bytes and EOF must remain authoritative to existing Node streams, while result files or polling introduce a second result owner. One IPC channel separates control from target stdio.

Parse a hidden runner argument in the public CLI or ship another Node executable. Rejected because either choice expands the public application grammar or distribution surface. A packaging-only bootstrap keeps one physical executable and two private logical entries.

Cache the complete Linux native decision without another reachability check. Rejected because user-manager availability can change between spawns. The runtime caches only the first successful deep bootstrap/scope probe and lightly rechecks manager reachability for every later eligible call.

Recover a failed native launch by replaying the command. Rejected because an ambiguous failure may occur after target execution and replay can therefore execute the command twice.

Infer startup failure from an unconsumed request alone. Rejected because abort, timeout, or disposal can terminate the launcher before request consumption. Matching an observed signal to an owner-issued termination preserves cancellation without hiding an unrelated bootstrap exit or recorded pre-exec failure.

Consequences

Supported Linux ordinary and PTY launches and Windows ordinary launches retain descendants through process-group escape and direct-parent exit, while direct target results remain independent from range quiescence. The cost is a per-spawn Linux manager check and scope/request or Windows runner/IPC/Job lifecycle, plus explicit failure when the selected owner cannot prove settlement.

Fallback hosts continue to run commands but carry a visible weaker guarantee. Windows ConPTY, macOS native containment, active breakaway descendants, old or absent user-systemd environments, target replay, persistent runner recovery, and termination paths where JavaScript cannot execute remain outside this decision.