deepseek-harness/.agents/notes/implemented/architecture/2026-08-27-outbound-proxy-policy.md

Agent Note: One outbound proxy policy, installed before anything can request

Status: implemented

English | 中文

Problem

Node's built-in fetch ignores HTTP_PROXY and HTTPS_PROXY. Every other tool a developer runs — curl, git, npm, pip — honours them, so a user behind a proxy exports the variables once and expects everything to follow. The harness did not: setGlobalDispatcher, ProxyAgent, and EnvHttpProxyAgent appeared zero times across packages/ and apps/, so the model request, every web search, web_fetch, MCP over HTTP, and the OTLP exporter all connected directly, silently, with no diagnostic anywhere.

The repository had briefly had an answer and lost it without noticing. PR #971 set NODE_USE_ENV_PROXY=1 in bin/dsh; eleven days later the “cleanup: remove managed source installer” change deleted that launcher wholesale, taking the flag with it. What survived was one sentence in apps/cli/reference/README.md telling the reader to set a variable that nothing consumed any more.

That sentence could not have worked anyway, for three measured reasons. NODE_USE_ENV_PROXY samples the environment at process start, while loadLayeredEnv() merges the .env layers afterwards, so a proxy declared in $DSH_HOME/.env is invisible to it. It reaches Node 24.0+ and, on the 22 line, only 22.21+ — the runtime floor at adoption was Node 22.19, where the variable did not exist and setting it warned about nothing. And it does not reach web-fetch-http at all: that provider passes its own dispatcher to fetch, and an explicit dispatcher overrides the global one whatever the flag says.

Decision

One policy, resolved once from the launch environment, installed as the global dispatcher. packages/util/http-proxy resolves a ProxyPolicy and installs it in runProfile immediately after the environment snapshot is provided and before any entry mounts. Node's fetch resolves undici's global dispatcher, so every plain fetch() and every SDK that reaches globalThis.fetch is covered without touching its code — nine call sites at the time of writing, and every future one for free. loadLayeredEnv has exactly one caller and apps/web ships no bin, so this single site covers every profile including sdk-minimal, which does not layer over base.

Resolution reads the launcher's snapshot rather than process.env, which is what makes a proxy in $DSH_HOME/.env work — the capability the environment-variable approach cannot have. Only that file: loadLayeredEnv refuses a proxy name in the project .env exactly as it refuses PATH or NODE_OPTIONS there, because that file arrives with a clone and must not choose the harness's route. The home file is exempt for the four proxy names alone, and DSH_HOME is itself bootstrap-only, so no .env can point the exemption at a directory a repository controls.

A library in util/, not a plugin. Transport policy has one answer per process: nothing to swap, and no scope narrower than the process to give one. The package exports functions and mounts nothing — boot, web, subprocess, and workflow all consume it, and util/ is the group every other group may depend on.

An earlier revision put it in a new net/ package group, reasoning that depending on undici disqualified it from a "zero-dependency" group. That reading was wrong: dependencies over hand-rolling records that the charter governs harness dependencies — util stays free of them so any group can depend on util — and does not ban external packages. What did need removing was the dependency on dsh-launch-environment: resolution needs one method from it, so it names a structural EnvLookup instead and the launcher passes its snapshot unchanged.

The plugin that revision shipped is gone with it. It let a composition declare the policy in cordis.yml, but no shipped bundle mounted it, so the launcher's path was the only reachable one — and its Config was the sole supplier of a configuration branch nothing else could reach.

One operation per caller need. installProxyFromEnvironment, proxyRouteFor, proxyEnvironmentForChild, and clearedProxyEnv cover installation, per-request routing, child inheritance and fixture isolation. SDK-specific factories would expose individual transport constraints through the shared API. web-fetch-http consumes the resolved route when constructing its pinning transport; the OTLP exporter remains direct. Installation includes resolution and diagnostic reporting because callers need one operation that resolves and installs routing.

proxyRouteFor also closes a defect the old accessor made expressible. web-fetch-http read the policy to decide whether to pin, then read it again to build a transport; an unmount between the two returned a direct, unpinned agent for a URL the first read had cleared as proxied. A route carries both, so the branch and the request cannot disagree. Its dispatcher is the process-wide one, closed rather than destroyed on disposal, so a request already in flight when a policy is unmounted still finishes.

The installed dispatcher routes by the policy, not by an environment it re-parses. Installation builds an Agent whose per-origin factory asks proxyForUrl where that origin goes, and returns a ProxyAgent or undici's own default client for it. undici's EnvHttpProxyAgent was the first choice and is wrong for this policy: when no HTTPS_PROXY is present it sets its HTTPS agent to the HTTP one, so a scheme this package keeps direct after refusing a SOCKS or malformed URL would still be tunnelled while the diagnostic said otherwise. Routing through the one predicate removes that class of divergence by construction rather than by test. Publishing the policy into the environment remains, but now serves one reader only: a spawned child, which has no policy object to consult.

This keeps proxyForUrl() and the dispatcher answering from one set of values. They must agree: if they disagreed about a URL, web-fetch-http would pin a connection the dispatcher meant to tunnel.

Resolution supplies what neither Node nor undici does. ALL_PROXY backs both schemes; a blank value counts as unset, because undici's ?? chain lets an empty lowercase name shadow a populated uppercase one; loopback is always bypassed, since the Web UI, the Connection transport, and every local test server would otherwise route through the proxy and loop. The bypass list carries ::1 and [::1]: undici's own matcher reads a bare ::1 as host : port 1 and never exempts it.

Rejection is quiet, and never reroutes the refused scheme. A slot the user filled and this package refused keeps that scheme direct rather than falling through to ALL_PROXY or the HTTP proxy, so the diagnostic and the route agree. A SOCKS URL, an unparseable string, or an unsupported scheme is reported on stderr and skipped — the variable may have been exported for other tools, and a typo in it must not stop the agent from starting. The environment is the only source, so no configuration surface exists where AGENTS.md's fail-loud rule would apply instead.

web_fetch uses the installed dispatcher. The provider sends every request through the process-wide dispatcher, so a proxied hop goes through the proxy, which performs the origin's DNS, and a hop the policy bypasses, which includes every loopback and every NO_PROXY entry, connects directly. The provider does not restrict destination addresses (web capability seam).

The URL-level policy is untouched: http(s) only, no embedded credentials, the length cap, and the cross-origin redirect refusal all still apply on every hop.

Ordinary child processes receive proxy policy; PTC program environments omit it. proxyEnvironmentForChild() merges into scrubbedParentEnv(). PTC also executes workflow scripts and excludes these settings from program environments because a proxy URL may carry user:password; direct program requests go direct.

The child keeps the user's own values, and that is what once broke it. Node parses HTTP_PROXY and HTTPS_PROXY under NODE_USE_ENV_PROXY before running the program and exits on any scheme other than http: or https:; a socks4:// kept for curl therefore ended every Node child — MCP servers, subagent CLIs, npm — before its first line, while this process had reported only that the scheme stayed direct. Measured on Node 24.17: socks4://, ftp://, and a malformed value all exit 1; socks5:// happens to be accepted there. The flag is now withheld whenever a value the child receives is one this package refused, so such a child connects directly and curl still reads the value it was kept for. Handing the child the resolved value instead would have kept Node proxied at the price of silently rewriting what the user set for another tool.

This accepts a documented seam. Such a context matches bypass entries by Node's rules, which differ from this package's in separators and IPv4-range support, and the flag exists only on Node 22.21+ and 24+.

SDK transports need independent verification. The OTLP exporter selects http-exporter-transport on Node, which posts through node:http and bypasses the global fetch dispatcher; a globalThis.fetch reference in its browser implementation does not prove Node routing. The E2B removal retires a second SDK transport integration without changing this requirement.

Telemetry stays direct on purpose. Routing it needs one of two things, and both cost more than the channel is worth. An http.Agent reads the environment through proxyEnv, which arrived in Node 22.21 and 24.5 — above the runtime floor at adoption, so Node 22.19, 22.20, and 24.0–24.4 stayed direct regardless, and the proxy package would have to keep a createNodeHttpAgent export for a path that worked on some runtimes. The Node 24.9 floor removes this API gap; routing still needs independent transport integration and must preserve compression. Replacing the transport with the SDK's fetch delegate covers every runtime, but that delegate has no compression, and the shipped base bundle enables gzip: a realistic OTLP batch measures 6.4x smaller with it. An attempt that refused exporter.compression instead broke every test that boots the shipped bundle, and one that gzipped at the serializer worked but put transport code in a telemetry plugin to keep it working.

Weighed against that, telemetry is the one outbound channel whose loss costs the user nothing: no tool, no model request, and no session depends on it, and an export that cannot connect is already dropped silently. A user behind a mandatory proxy is left exactly where they were before this change rather than regressed. egress.spec.ts now asserts the exclusion — an SDK upgrade that moved the exporter onto fetch would start routing telemetry through a proxy silently, and that case is what makes it visible.

Every call site carries an egress test. Each owning package's egress.spec.ts drives its actual transport through a fake proxy and checks the observed route. These tests cover search backends, pi-ai discovery, MCP over HTTP, child Node processes, worker threads and telemetry's direct-route exception. They detect dependency transport changes that a static call-site check cannot observe.

A gate keeps the defect from returning. verify-no-bare-dispatcher parses the TypeScript AST — scripts/AGENTS.md requires syntax-aware discovery, and a line-wise regex missed both the { dispatcher } shorthand this repository already uses and a new Alias(...) behind a renamed import. It rejects an undici agent construction and an explicit dispatcher option outside the owning package. proxyRouteFor(url) is the sanctioned replacement, and a call site that genuinely owns its transport says so with a proxy-exempt: comment. The rule exists because web-fetch-http's original new Agent was entirely reasonable when it was written — proxying simply did not exist yet, and nothing would have caught it.

Alternatives considered

Document NODE_USE_ENV_PROXY=1 and stop. Rejected at adoption on the three measurements above: invisible to $DSH_HOME/.env, absent on the Node 22.19 floor, and bypassed by web-fetch-http regardless. The environment-snapshot and explicit-dispatcher limitations remain on the current runtime floor.

Thread a policy value to every call site. DeepSeek-Reasonix does this across 98 sites, buying a per-provider opt-out. Rejected: that opt-out exists for a need this harness does not have, and nine sites changed by hand means the tenth is forgotten — Pi's changelog records OAuth and Bedrock as two separate after-the-fact fixes of exactly that kind. The isolation argument for it is real, and is answered instead by handling worker threads explicitly and by proving disposal restores the previous dispatcher.

http.setGlobalProxyFromEnv(). Node's own programmatic switch covers fetch and node:http together and returns a restore function — the shape ctx.effect() wants. Unusable: added: v24.14.0, with nothing on the 22 line. Worth revisiting if engines ever rises past it.

Patch globalThis.fetch via undici.install(). Pi does, to keep fetch and the dispatcher on one undici when a newer Node's bundled fetch mishandles compressed responses through a userland dispatcher. Rejected as speculative here: this repository's engines ceiling has not reached that runtime.

Make this a capability seam. Rejected. Service Definition / Provider / Consumer is for swappable backends; this has one implementation and one answer per process. If operating-system proxy or PAC support ever lands, resolveProxyPolicy is the extension point.

Read the operating system's proxy settings. Rejected for this change. Only Codex and Reasonix among six surveyed products do it, and Codex keeps it behind a default-off flag. Measured on the author's machine, it would have found nothing: the proxy application had written the setting to the Wi-Fi service while the primary interface was a USB ethernet adapter with no proxy, so scutil --proxy reported none while the exported variables worked. It also needs its own bypass matcher, because an operating system list carries CIDR entries that neither undici nor Node matches.

Give model-authored code the proxy too. Rejected because a proxy URL may carry credentials. PTC processes, including workflow execution, keep those settings outside the program environment; direct network use remains subject to the program's execution policy.

Consequences

A user who exports HTTPS_PROXY, or writes it into $DSH_HOME/.env, is proxied everywhere the harness makes a request, with no flag and no configuration. The launcher installs it exactly once, before the first plugin mounts.

Because the operating system's settings are not read, the user-facing documentation is now load-bearing rather than supplementary: a user who only toggled "system proxy" in a proxy application gets nothing and no diagnostic. docs/user/guide/network-proxy.md therefore states which variables to export and why a browser is proxied when a terminal is not — the three-mechanism confusion is the single most common report, and it is not specific to this harness.

Reaching Node's built-in fetch from a userland undici depends on both writing the legacy Symbol.for('undici.globalDispatcher.1') slot. That is an implicit cross-version coupling rather than a contract — corepack#834 records it breaking — so tests/install.spec.ts drives a real request through a loopback proxy. A version bump that breaks the coupling fails there instead of in the field.

The suite is hermetic against the developer's own environment: every Vitest configuration runs scripts/test-proxy-environment.ts, which clears all eight proxy names in both casings before any test, and install.spec.ts restores the machine's values around each case that sets its own. It has to. An exported lowercase all_proxy decided a test's outcome during development, because resolution reads lowercase first.

Testing

packages/util/http-proxy holds 84 tests at 100% per-file coverage. Resolution covers precedence, the ALL_PROXY fallback, blank-shadowing, the SOCKS and malformed diagnostics, and the HTTPS-only environment that leaves http: direct; routing covers the whole loopback range structurally, and bypass matching covers suffixes, ports, both IPv6 spellings, and the CIDR entry that deliberately does not match. Installation drives a real loopback proxy and asserts the absolute-form request arrives, that a bypassed target does not, and that disposal restores the dispatcher, the policy, and the environment. Every case installs through installProxyFromEnvironment, so no test can assert a policy object a real launch could not produce.

packages/web/web-fetch-http/tests/proxy.spec.ts asserts that a proxied hop reaches the proxy, a bypassed hop connects directly, and the cross-origin redirect refusal survives on the proxied path.

verify-no-bare-dispatcher.spec.ts proves the gate rejects the exact shape this package was introduced to fix, accepts proxyRouteFor, accepts an annotated exemption, and passes on the current tree.

The egress suite carries telemetry's case in the negative: the shipped backend exports under an installed policy and the fake proxy sees nothing, so the deliberate exclusion is asserted rather than merely documented. A parity suite checks proxyForUrl against where a real fetch actually went for every form in the documented NO_PROXY vocabulary; since the dispatcher routes by that same predicate, what it now catches is a form bypassesProxy reads differently from how the vocabulary documents it, and any future dispatcher that reintroduces a second matcher.

No recorded-session snapshot changes: nothing here alters a model-visible input or product-user-visible transcript output.