Files
deepseek-harness/packages/util/output-retention
Turtle 38c7a9fbf4 feat(release): publish package source, declaration maps, and documentation (#5884)
* feat(release): publish package source, declaration maps, and documentation

Every release member publishes its TypeScript source and the declaration
maps that resolve into it, and @deepseek-ai/dsh publishes the repository
Markdown documentation staged from docs/ at prepack time.

The ./src/* export most members already declared now resolves for an
installed consumer, and a consumer editor follows a declaration to the
implementation. Bundled JavaScript maps stay unpublished; the vendored
family and the Desktop runtime file policy are unchanged.

Rationale: .agents/notes/implemented/process/2026-10-08-published-source-and-docs.md

* feat(release): publish JavaScript maps and compress bundles

Every emitted JavaScript map ships beside the bundle it describes, and the
workspace build and the client preset compress bundles without mangling
identifiers.

The documentation payload now mirrors the website publication manifest and
is removed again by `postpack`; Desktop and the Python executable exclude
the published source, documentation, and maps; `clean` removes a staged
copy. Typert's publication contract gains the Remote declaration map.

* fix(release): keep node bundles readable for the worker image

The browser worker image lowers `yield*` only as a whole statement and
rewrites `for await` loops by their source layout, so compressed node-side
entries either fail that lowering or lower to unparseable code. Only the
browser bundles keep compression, which removes about a third of their text.

The bundle-purity assertion now matches the module-loader call regardless of
the quote style compression picks, and the release-pack fixture exercises
the installed package's new documentation requirement.

* feat(release): publish the whole documentation tree

`@deepseek-ai/dsh` copies `docs/` in full instead of the website's page
set: the maintainer documents, the generated migration records, the
translation guidelines, and the `*.i18n.yaml` consistency records ship
with the installed package, so the documentation's own links resolve
without a checkout.

* feat(release): publish the upgrade guides with the documentation

The staged documentation keeps the website's page set and adds every
`docs/upgrade-guide/` document, which is what an installed reader consults
when a released change breaks something. The rest of the maintainer tree
stays out.

* test(release): expect source and maps in master's new payload cases

* feat(release): keep installed docs and source readable in Desktop

Desktop keeps the published src/ and docs/ trees inside app.asar, and
every model shell call receives DSH_INSTALL_DIR and DSH_NODE_EXECUTABLE
so an agent can read them through Electron's Node mode. Staging also
generates docs/upgrade-guide/README.md, an index of the upgrade guides
grouped by the release they upgrade from.

Minified client bundles print string literals with backticks: the
worker image packer now recognizes those chunk requests, client license
banners are prepended after minification so their text survives, and
the affected artifact tests accept either quote.

* test(desktop): read archived docs through the Windows PowerShell form

Regenerate the config catalog for the shifted shell-env source line.

* test(desktop): wait for the GUI-subsystem Electron process from PowerShell

* build(release): publish source and maps from the message-quotes package

* build(release): publish source and maps from packages new on master

* build(release): publish source and maps from the tool-office package
2026-11-05 00:22:36 +08:00
..

description, kind
description kind
Bounded model-facing output for tools that must cap how much context they return: item and text retainers plus a standardized omission footer. package-library

@deepseek-ai/dsh-output-retention

English | 中文

Summary

Use dsh-output-retention to cap the items or text a tool returns to a model while reporting what was omitted. ItemRetainer keeps an ordered head window and can report an exact omitted-item count; TextRetainer keeps head, tail, or head-and-tail byte windows without returning invalid UTF-8 cuts. formatRetentionNotice adds a consistent omission clause while each tool supplies its own recovery guidance. truncateWithoutSplittingSurrogatePair caps a character-budget preview without leaving a lone high surrogate at the cut. Grouping, line numbering, spill files, and provider errors remain tool responsibilities; consumers import this library directly rather than loading it through cordis.yml.

Table of Contents


Use this package

Use a retainer wherever a tool must cap how much of its result reaches the model, and report honestly what was dropped. Choose ItemRetainer for ordered logical units and TextRetainer for byte-oriented streams.

Bounding a list of items

import { ItemRetainer } from '@deepseek-ai/dsh-output-retention'

declare const globMaxResults: number
declare const candidates: AsyncIterable<{ path: string }>
const retainer = new ItemRetainer<{ path: string }>({ kind: 'head', maxItems: globMaxResults })
for await (const entry of candidates) {
  retainer.push(entry)          // keep draining past the cap for an exact count
}
const { items, truncated, omitted } = retainer.finish()

push() reports per item whether it was kept, and finish() returns the retained items plus omitted — an exact count when the caller kept feeding every observed unit. A search tool can collect the full result set for a spill file while retaining only the first page for the model.

Bounding a text stream

import { TextRetainer } from '@deepseek-ai/dsh-output-retention'

const out = new TextRetainer({ kind: 'headTail', headBytes: headCap, tailBytes: tailCap })
child.stdout.on('data', (chunk: Buffer) => { out.push(chunk) })
const { text, omittedBytes } = out.finish()

head, tail, and headTail count bytes, not characters or lines: a child's pipe and an HTTP body are byte streams. finish() trims a partial codepoint at each cut, so the returned text never carries a replacement character introduced by the cut, and a codepoint is never reconstructed across the omitted middle.

import { formatRetentionNotice } from '@deepseek-ai/dsh-output-retention'

declare const grepMaxMatches: number
declare const items: { length: number }
import type { Omitted } from '@deepseek-ai/dsh-output-retention'

declare const omitted: Omitted

const footer = formatRetentionNotice(
  { scope: 'grep', strategy: 'head', unit: 'items', limit: grepMaxMatches, kept: items.length, omitted },
  ({ kept }) => `Results capped at ${kept}. Narrow the pattern, path, or include to see more.`,
)

The library standardizes the omission clause (Omitted 3 items.) and joins it with the tool's own recovery guidance; only the tool knows the recovery action, so the tool supplies those words.

Capping a character-budget preview

truncateWithoutSplittingSurrogatePair caps a preview measured in UTF-16 code units. A cut that lands inside a surrogate pair drops the unpaired half instead of returning a lone high surrogate, so the kept text stays a prefix. The persistent bash and pwsh shells and str_replace_editor use it for their character-budget previews; each still owns its truncation notice and its incomplete prefix.

What truncated means

truncated is a budget fact: the retainer omitted otherwise-available content because of a cap. It never means the upstream was incomplete — permission failures, skipped binary files, provider partial failures, and unreadable candidates stay in tool-domain fields, never folded into truncated.

How the current tools use it

Tool Retainer What the tool still owns
glob ItemRetainer, head Spill-file collection, path mapping, skipped candidates, incomplete
grep ItemRetainer, head Spill-file collection, per-match preview truncation, grouping, sorting
bash TextRetainer, tail or headTail Spill files, exit status, signal, timeout, background jobs
web_fetch TextRetainer, head or headTail Provider and resource caps, error states
web_search ItemRetainer, head The "sources capped" notice wording and provider facts

read stays outside this library: its line-window pagination (offset/limit, line numbers, totalLines) is a file-specific renderer that a single omission count cannot represent.


Understand the implementation

Implementation internals — click to expand

The library is built on one separation: it owns the mechanical question of what was kept and what was omitted; tool packages own every business meaning.

Source map

File Role
src/index.ts ItemRetainer, TextRetainer, describeOmitted, formatRetentionNotice, and truncateWithoutSplittingSurrogatePair

Two retainers, two resource models

ItemRetainer bounds ordered logical units and keeps only the first maxItems; the caller keeps pushing every observed unit so the omission count is exact. TextRetainer bounds bytes with one shared prefix/suffix accumulator: head is prefix-only, tail is suffix-only, headTail is both, and the accumulator holds at most headBytes + tailBytes + one chunk in memory, so a large stream does not accumulate unbounded.

How the budget facts stay honest

push() returns kept (this unit or chunk fully retained) and truncated (anything dropped yet). finish() reports omission against the bytes actually returned, so a UTF-8 boundary trim that drops partial-codepoint bytes is counted too — a notice built from the budget alone would overstate the retained text. describeOmitted prints a count only for exact; unknown prints no count because the caller provided none.

The read-render exclusion

read's offset/limit pagination is a line-window renderer with its own byte cap over the selected window; a single Omitted value cannot represent both sides of that window, so it stays out of this library.


Further Exploration

Read these pages when you need the consumers or the boundary decision behind the library.

  • Spill policy — composes TextRetainer for a bounded preview around a spill-file notice.
  • Spill subsystem — the spill vocabulary this library's preview mechanics serve.
  • File search tool — an ItemRetainer consumer collecting full results for spill.

Model Experience

Indirectly, through the retention consumers that render retained content and omission metadata.

KV Cache effect

No direct invalidation; the retention consumers own any request-prefix changes.

Known Limitations and Deferred Work

These limits define what the retainers deliberately do not cover. They are current package constraints, not a task backlog.

  • Item retention supports head only — tail, head/tail, pagination, grouping, and provider-completeness semantics remain tool-owned.
  • Text retention is byte-oriented — TextRetainer counts bytes; character-budget head caps go through truncateWithoutSplittingSurrogatePair, and line windows such as read pagination still require a separate renderer. A cut may discard partial UTF-8 boundary bytes to keep the returned text valid.

Dev Note

Working context for maintainers — click to expand

None.