refactor(api): move Remote result projection to gateway

This commit is contained in:
Yudong Han authored and yudshj committed 2026-09-21 12:50:42 +08:00
1 parent 2aac7c8ef5
commit e98b5703c7
22 files changed
+138 -202

No files matched your search

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-02-typert-remote-method-calls.md
2026-08-02-typert-remote-method-calls.md: 9986e0dac5e85d0aadc745ca4084ed3d1cecae54
2026-08-02-typert-remote-method-calls.zh.md: 282af768413498658722af3adee17fb59a8e93c2
2026-08-02-typert-remote-method-calls.md: 2a12aaf78f638d8af786d2ca817a3fee07a2c614
2026-08-02-typert-remote-method-calls.zh.md: bf4a7b0179b4dc878a0a53c7076c02e6f1bf0273
@@ -31,8 +31,8 @@ The Remote consumer projection contains `.d.ts`, `.d.ts.map`, and `.js` files. T
| `@deepseek-ai/dsh-typert-protocol` | Declares only the minimal `ctx.typert` protocol | `TypertRemoteService`, decorators, binding fallback, descriptors, lookup/Context, and the Remote map; no dependency on the compiler, Zod, Connection, or Browser |
| Typert registry | `ctx.typert` | Separately stores reflection for the current environment, imported Remote contributions, lookup providers, and Context providers |
| Typert generator/loader | No new business service | Generates three kinds of `lib` artifacts from the Host/Client Programs and registers the current environment's artifacts with `ctx.typert` |
| API Gateway's Host face | `ctx.typertGateway` | Associates Host definitions with live Services, decodes parameters, resolves receivers, and invokes methods |
| Connection | `ctx.connection` | Exclusively owns the HTTP Server/future WebSocket, the shared `/api` route, RPC envelope, rpcId, serialization, trust, error transport, Typert interception, and owner-registered exact Fetch routes on the same channel |
| API Gateway's Host face | `ctx.typertGateway` | Associates Host definitions with live Services, decodes parameters, resolves receivers, invokes methods, and projects unary results into carrier-neutral fields and attachments |
| Connection | `ctx.connection` | Exclusively owns the HTTP Server/future WebSocket, the shared `/api` route, RPC envelope, rpcId, physical serialization, trust, error transport, Typert interception, and owner-registered exact Fetch routes on the same channel |
| API Gateway's Client face | `ctx.remote`, `ctx.remote.<namespace>` | Mounts Remote contributions, materializes each namespace as a traced `remote.<namespace>` child Service, and delegates canonical calls to `ctx.connection.rpc` |
| API Remotes | No new service | Owns Host Agent/Session lookup policy and serves as the only Client business facade, selecting and mounting `/remote` contributions while exposing the selected API declarations |
| Agent/Session owning packages | Existing domain services | Provide both static interface merges and runtime lookup/Context providers |
@@ -31,8 +31,8 @@ Remote 消费端投影同时包含 `.d.ts`、`.d.ts.map` 和 `.js`。`.d.ts` 只
| `@deepseek-ai/dsh-typert-protocol` | 只声明 `ctx.typert` 的最小协议 | `TypertRemoteService`、decorator、binding 回退、descriptor、lookup/Context 和 Remote map;不依赖 compiler、Zod、Connection 或 Browser |
| Typert registry | `ctx.typert` | 分开保存当前环境 reflection、导入的 Remote contribution、lookup provider 和 Context provider |
| Typert generator/loader | 无新增业务服务 | 从 Host/Client Program 生成三类 `lib` 产物,并把当前环境产物注册到 `ctx.typert` |
| API Gateway 的 Host face | `ctx.typertGateway` | 关联 Host definition 与活 Service,解码参数、解析 receiver 并调用方法 |
| Connection | `ctx.connection` | 独占 HTTP Server/未来 WebSocket、共享 `/api` route、RPC envelope、rpcId、序列化、trust、错误传输、Typert 拦截,以及各 owner 在同一 channel 上注册的精确 Fetch route |
| API Gateway 的 Host face | `ctx.typertGateway` | 关联 Host definition 与活 Service,解码参数、解析 receiver、调用方法,并将一元结果投影为载体无关的字段与附件 |
| Connection | `ctx.connection` | 独占 HTTP Server/未来 WebSocket、共享 `/api` route、RPC envelope、rpcId、物理序列化、trust、错误传输、Typert 拦截,以及各 owner 在同一 channel 上注册的精确 Fetch route |
| API Gateway 的 Client face | `ctx.remote`、`ctx.remote.<namespace>` | mount Remote contribution,把每个 namespace 实体化为可追踪的 `remote.<namespace>` 子 Service,并把规范调用交给 `ctx.connection.rpc` |
| API Remotes | 无新增服务 | 负责 Host Agent/Session lookup 策略,并作为 Client 业务的唯一 facade,选择并挂载 `/remote` contribution,同时暴露所选 API 声明 |
| Agent/Session owning 包 | 既有领域服务 | 同时提供静态 interface merge 与运行时 lookup/Context provider |
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-17-workspace-file-binary-transfer.md
2026-09-17-workspace-file-binary-transfer.md: 1cfffb4113619fba17479065bac6bb420f214d7d
2026-09-17-workspace-file-binary-transfer.zh.md: 1cacc2c294b979ed7e840c84293eb3f6421c6bcb
2026-09-17-workspace-file-binary-transfer.md: b419c8bd15f51246825ceddf2905ec055bbc70ae
2026-09-17-workspace-file-binary-transfer.zh.md: 9cf3d7074c3d536a5c8a39732231c6996d6f6ead
@@ -12,7 +12,7 @@ Document previews need native bytes. Encoding file contents as base64 adds about
The [Remote method architecture](2026-08-02-typert-remote-method-calls.md) owns method dispatch and lifetime; this note defines binary success encoding and Client validation. The [workspace file service](2026-09-05-workspace-files-service.md) exposes one binary `readBytes` Remote. Its required options object independently selects a byte `range` and a `baseFile` for relative target resolution. Without `range`, it reads a complete file. This avoids separate method names for orthogonal choices while preserving bounded `fs.readBytes` and `fs.readByteRange` reads.
Typert recursively recognizes `Uint8Array` in unary result types, including root values, optional fields, and containers. Generated result codecs provide `encode()` for subtrees whose types can contain bytes and `decode()` to validate the reconstructed value without iterating, copying, or freezing byte payloads; Client declarations narrow each byte buffer to `ArrayBuffer`. Gateway supplies the encoder to Connection, using identity for strict JSON results so they avoid recursive detection. Source-mode calls retain runtime byte detection. Connection owns standard `FormData` parts and preserves the RPC envelope and correlation id in JSON metadata; it does not depend on Typert reflection. An attachment table associates byte parts with paths through the JSON result; field names remain business-owned. The Client restores `Uint8Array` views over `Blob.arrayBuffer()` and Gateway delegates validation to the generated decoder. Ordinary results and errors retain JSON responses. Gateway owns method dispatch, Session lookup, cancellation, and mount lifetime; feature code needs no multipart reader or dedicated route.
Typert recursively recognizes `Uint8Array` in unary result types, including root values, optional fields, and containers. Generated result codecs provide `encode()` for subtrees whose types can contain bytes and `decode()` to validate the reconstructed value without iterating, copying, or freezing byte payloads; Client declarations narrow each byte buffer to `ArrayBuffer`. Gateway executes the generated encoder, keeps strict JSON values unchanged, and performs runtime byte detection for source-mode calls. It returns JSON-compatible metadata and result-relative byte attachments to Connection. Connection owns standard `FormData` parts and preserves the RPC envelope and correlation id in JSON metadata; it does not inspect business values or depend on Typert reflection. An attachment table associates byte parts with paths through the JSON result; field names remain business-owned. The Client restores `Uint8Array` views over `Blob.arrayBuffer()` and Gateway delegates validation to the generated decoder. Ordinary results and errors retain JSON responses. Gateway owns method dispatch, Session lookup, cancellation, result projection, and mount lifetime; feature code needs no multipart reader or dedicated route.
## Alternatives considered
@@ -12,7 +12,7 @@ Status: implemented
[Remote 方法架构](2026-08-02-typert-remote-method-calls.zh.md)负责方法分发与生命周期;本记录规定二进制成功响应的编码及 Client 校验。[工作区文件服务](2026-09-05-workspace-files-service.zh.md)提供单一的二进制 `readBytes` Remote。其必填选项对象独立选择字节 `range` 和用于相对目标解析的 `baseFile`。未传 `range` 时读取完整文件。这避免为正交的选择设置不同方法名,同时保留有界的 `fs.readBytes` 和 `fs.readByteRange` 读取。
Typert 递归识别一元结果类型中的 `Uint8Array`,包括根值、可选字段和容器。生成的结果 codec 提供 `encode()` 处理类型可能包含字节的子树,并通过 `decode()` 校验还原后的值,不逐字节遍历、复制或冻结字节载荷;Client 声明将每个字节缓冲区限定为 `ArrayBuffer`。Gateway 将 encoder 交给 Connection,对 strict JSON 结果原值传递,从而避免递归识别。源码模式调用保留运行时字节识别。Connection 负责标准 `FormData` 部分,并在 JSON 元数据中保留 RPC 信封和关联 id,不依赖 Typert 反射。附件表将字节部分与 JSON 结果中的路径关联,字段名仍由业务拥有。Client 恢复基于 `Blob.arrayBuffer()` 的 `Uint8Array` 视图,Gateway 将校验交给生成的解码器。普通结果与错误继续使用 JSON 响应。Gateway 负责方法分发、Session 查找、取消及挂载生命周期;功能代码无需 multipart 读取器或专用路由。
Typert 递归识别一元结果类型中的 `Uint8Array`,包括根值、可选字段和容器。生成的结果 codec 提供 `encode()` 处理类型可能包含字节的子树,并通过 `decode()` 校验还原后的值,不逐字节遍历、复制或冻结字节载荷;Client 声明将每个字节缓冲区限定为 `ArrayBuffer`。Gateway 执行生成的 encoder,保持 strict JSON 值不变,并为源码模式调用执行运行时字节识别;随后向 Connection 返回 JSON 兼容元数据及相对于结果的字节附件。Connection 负责标准 `FormData` 部分,并在 JSON 元数据中保留 RPC 信封和关联 id;它不检查业务值,也不依赖 Typert 反射。附件表将字节部分与 JSON 结果中的路径关联,字段名仍由业务拥有。Client 恢复基于 `Blob.arrayBuffer()` 的 `Uint8Array` 视图,Gateway 将校验交给生成的解码器。普通结果与错误继续使用 JSON 响应。Gateway 负责方法分发、Session 查找、取消、结果投影及挂载生命周期;功能代码无需 multipart 读取器或专用路由。
## 考虑过的替代方案
+1 -1
View File
@@ -122,7 +122,7 @@ Strict analysis requires a Remote to be a public, non-static instance method wit
Remote calls use the Connection's `/api` route. The Client Remote calls `connection.rpc.call('/api', '<namespace>/<method>', { args }, signal)`; the HTTP carrier maps this to `POST /api/<namespace>/<method>`, with a payload containing only a named `args` object.
The Connection performs the unified trust check for `/api` before the HTTP bridge, then dispatches inside the shared FetchHandler. The Typert Gateway claims only two-segment endpoints that have a strict descriptor or active SRC marker; binary Remote successes use multipart responses, and feature-owned exact Fetch routes handle responses outside the RPC envelope, and other requests return 404. The Connection owns transport, RPC ids, response envelopes, and request cancellation, while the Gateway owns only the Remote data protocol and business dispatch. Replacing the Connection carrier does not require changes to Remote descriptors or the Client programming interface.
The Connection performs the unified trust check for `/api` before the HTTP bridge, then dispatches inside the shared FetchHandler. The Typert Gateway claims only two-segment endpoints that have a strict descriptor or active SRC marker; it projects binary Remote fields into JSON-compatible metadata and result-relative byte attachments, which Connection frames as multipart responses. Feature-owned exact Fetch routes handle responses outside the RPC envelope, and other requests return 404. Connection owns transport, RPC ids, response envelopes, and request cancellation, while Gateway owns the Remote data protocol and business dispatch. Replacing the Connection carrier does not require changes to Remote descriptors or the Client programming interface.
For every call, the Gateway resolves the descriptor and live service from the current registries instead of caching business objects. It requires the fields in `args` to match the descriptor exactly, validates wire values with codecs, resolves objects or receivers through registered lookup or Context providers, and invokes the service method targeted by the binding. The Client Gateway validates decoded binary results with the generated result codec; ordinary JSON results retain their existing handling. A missing provider, unknown identity, binding mismatch, missing or extra argument, schema failure, or missing method fails before entering or after leaving business code.
+1 -1
View File
@@ -122,7 +122,7 @@ Remote Client 声明中的参数名来自 wire 字段,参数和返回类型则
Remote 调用使用 Connection 的 `/api` 路由。Client Remote 调用 `connection.rpc.call('/api', '<namespace>/<method>', { args }, signal)`;HTTP carrier 对应 `POST /api/<namespace>/<method>`,payload 只包含一个具名 `args` 对象。
Connection 在 HTTP bridge 之前执行 `/api` 的统一信任检查,再在共享 FetchHandler 内分发。Typert Gateway 只认领存在严格描述符或活跃 SRC marker 的两段式 endpoint;二进制 Remote 成功响应使用 multipart,功能自有的精确 Fetch 路由处理 RPC 信封之外的响应,其他请求返回 404。Connection 拥有传输、RPC id、响应 envelope 和请求取消,Gateway 只拥有 Remote 数据协议和业务分发。替换 Connection carrier 不要求改变 Remote 描述符或 Client 编程接口。
Connection 在 HTTP bridge 之前执行 `/api` 的统一信任检查,再在共享 FetchHandler 内分发。Typert Gateway 只认领存在严格描述符或活跃 SRC marker 的两段式 endpoint;它把二进制 Remote 字段投影为 JSON 兼容元数据及相对于结果的字节附件,再由 Connection 封装为 multipart 响应。功能自有的精确 Fetch 路由处理 RPC 信封之外的响应,其他请求返回 404。Connection 拥有传输、RPC id、响应 envelope 和请求取消,Gateway 拥有 Remote 数据协议和业务分发。替换 Connection carrier 不要求改变 Remote 描述符或 Client 编程接口。
Gateway 每次调用都从当前注册表解析描述符和实时服务,不缓存业务对象。它要求 `args` 的字段集合与描述符完全一致,先用 codec 校验 wire 值,再通过注册的 lookup 或 Context 提供方解析对象或接收者,最后调用 binding 指向的服务方法。Client Gateway 使用生成的结果 codec 校验解码后的二进制结果;普通 JSON 结果保持原有处理方式。缺少提供方、identity 未命中、binding 不一致、参数缺失或多余、schema 失败和方法不存在都会在进入业务代码前或离开业务代码后失败。
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/config-catalog.md
config-catalog.md: 645c64b43a95d227a9373985f05bc50b4b4e93f6
config-catalog.zh.md: e9f8db259539b83f771027062019039b930f8d37
config-catalog.md: 1fac327a8a50a09cf694f6fc31a0af1f031f596f
config-catalog.zh.md: fbc043ab62d57d1b7f754e28d3469cc5ebd4fb49
+2 -2
View File
@@ -199,7 +199,7 @@ export interface Config {
}
```
Source: [`packages/api/gateway/src/index.ts:140`](../packages/api/gateway/src/index.ts)
Source: [`packages/api/gateway/src/index.ts:144`](../packages/api/gateway/src/index.ts)
<a id="deepseek-aidsh-api-job-controller"></a>
@@ -466,7 +466,7 @@ export interface ConnectionRecoveryConfig {
}
```
Source: [`packages/client/connection/src/index.ts:90`](../packages/client/connection/src/index.ts)
Source: [`packages/client/connection/src/index.ts:92`](../packages/client/connection/src/index.ts)
<a id="deepseek-aidsh-client-hmr"></a>
+2 -2
View File
@@ -201,7 +201,7 @@ export interface Config {
}
```
来源:[`packages/api/gateway/src/index.ts:140`](../packages/api/gateway/src/index.ts)
来源:[`packages/api/gateway/src/index.ts:144`](../packages/api/gateway/src/index.ts)
<a id="deepseek-aidsh-api-job-controller"></a>
@@ -470,7 +470,7 @@ export interface ConnectionRecoveryConfig {
}
```
来源: [`packages/client/connection/src/index.ts:90`](../packages/client/connection/src/index.ts)
来源: [`packages/client/connection/src/index.ts:92`](../packages/client/connection/src/index.ts)
<a id="deepseek-aidsh-client-hmr"></a>
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/event-producer-consumer.md
event-producer-consumer.md: de767c7629adcf554b3c58e43bb91af62e7aac1c
event-producer-consumer.zh.md: a57e1b7e9447939e6a6586c4743dfd39345cc15c
event-producer-consumer.md: 2b373821a2d0c9acf2284c4195a928947700f1ce
event-producer-consumer.zh.md: 90ca2a4c76ac477265a60dad80c5f65d04509e90
+1 -1
View File
@@ -30,7 +30,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac
| `authorization/settled` | `emit` | [`packages/credentials/authorization/src/index.ts:57`](../packages/credentials/authorization/src/index.ts) | [`authorization`](../packages/credentials/authorization) (`events.dispatch`) | [`authorization`](../packages/credentials/authorization) |
| `commands/change` | `emit` | [`packages/interaction/commands/src/types.ts:89`](../packages/interaction/commands/src/types.ts) | [`commands`](../packages/interaction/commands) (`events.dispatch`) | `remotes` |
| `compaction/summary-error` | `waterfall` | [`packages/compaction/compaction/src/index.ts:106`](../packages/compaction/compaction/src/index.ts) | [`compaction-basic`](../packages/compaction/compaction-basic) (`waterfall`) | [`compaction-image-offload`](../packages/compaction/compaction-image-offload) |
| `connection/request` | `waterfall` | [`packages/client/connection/src/index.ts:65`](../packages/client/connection/src/index.ts) | `connection` (`waterfall`) | - |
| `connection/request` | `waterfall` | [`packages/client/connection/src/index.ts:67`](../packages/client/connection/src/index.ts) | `connection` (`waterfall`) | - |
| `cordis/dynamic-package` | `emit` | [`packages/extensions/cordis-host-runner/src/types.ts:380`](../packages/extensions/cordis-host-runner/src/types.ts) | [`cordis-host-runner`](../packages/extensions/cordis-host-runner) (`emit`) | `remotes` |
| `cordis/dynamic-retract` | `emit` | [`packages/extensions/cordis-host-runner/src/types.ts:386`](../packages/extensions/cordis-host-runner/src/types.ts) | [`cordis-host-runner`](../packages/extensions/cordis-host-runner) (`emit`) | `remotes` |
| `cordis/inspect-query` | `emit` | [`packages/extensions/cordis-host-runner/src/types.ts:392`](../packages/extensions/cordis-host-runner/src/types.ts) | [`cordis-host-runner`](../packages/extensions/cordis-host-runner) (`emit`) | `remotes` |
+1 -1
View File
@@ -32,7 +32,7 @@
| `authorization/settled` | `emit` | [`packages/credentials/authorization/src/index.ts:57`](../packages/credentials/authorization/src/index.ts) | [`authorization`](../packages/credentials/authorization) (`events.dispatch`) | [`authorization`](../packages/credentials/authorization) |
| `commands/change` | `emit` | [`packages/interaction/commands/src/types.ts:89`](../packages/interaction/commands/src/types.ts) | [`commands`](../packages/interaction/commands) (`events.dispatch`) | `remotes` |
| `compaction/summary-error` | `waterfall` | [`packages/compaction/compaction/src/index.ts:106`](../packages/compaction/compaction/src/index.ts) | [`compaction-basic`](../packages/compaction/compaction-basic) (`waterfall`) | [`compaction-image-offload`](../packages/compaction/compaction-image-offload) |
| `connection/request` | `waterfall` | [`packages/client/connection/src/index.ts:65`](../packages/client/connection/src/index.ts) | `connection` (`waterfall`) | - |
| `connection/request` | `waterfall` | [`packages/client/connection/src/index.ts:67`](../packages/client/connection/src/index.ts) | `connection` (`waterfall`) | - |
| `cordis/dynamic-package` | `emit` | [`packages/extensions/cordis-host-runner/src/types.ts:380`](../packages/extensions/cordis-host-runner/src/types.ts) | [`cordis-host-runner`](../packages/extensions/cordis-host-runner) (`emit`) | `remotes` |
| `cordis/dynamic-retract` | `emit` | [`packages/extensions/cordis-host-runner/src/types.ts:386`](../packages/extensions/cordis-host-runner/src/types.ts) | [`cordis-host-runner`](../packages/extensions/cordis-host-runner) (`emit`) | `remotes` |
| `cordis/inspect-query` | `emit` | [`packages/extensions/cordis-host-runner/src/types.ts:392`](../packages/extensions/cordis-host-runner/src/types.ts) | [`cordis-host-runner`](../packages/extensions/cordis-host-runner) (`emit`) | `remotes` |
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/typert.md
typert.md: a8ad968b7af77c90cfcdbeb910f435be461bf442
typert.zh.md: d020b75ee7c52cc87aaea371978aeebe78d6fa45
typert.md: db8b48945e2301e14d866b8a2e3df59c4ae408c9
typert.zh.md: cdaee66201d0f27a5a18aa99cb1888438051e018
+1 -1
View File
@@ -55,7 +55,7 @@ type TypertCodec =
*/
readonly decode?: (value: unknown) => unknown
/**
* Project typed binary fields into carrier-owned multipart attachments.
* Project typed binary fields into RPC result attachments.
* @param value - native unary result.
* @param writeBytes - records a byte view at its result-relative path and returns its JSON placeholder.
* @returns JSON metadata with untouched JSON subtrees retained.
+1 -1
View File
@@ -55,7 +55,7 @@ type TypertCodec =
*/
readonly decode?: (value: unknown) => unknown
/**
* Project typed binary fields into carrier-owned multipart attachments.
* Project typed binary fields into RPC result attachments.
* @param value - native unary result.
* @param writeBytes - records a byte view at its result-relative path and returns its JSON placeholder.
* @returns JSON metadata with untouched JSON subtrees retained.
+2
View File
@@ -21,8 +21,10 @@ export type {
ConnectionIndexRequest,
ConnectionIndexResponse,
ConnectionRpcEndpointMatcher,
ConnectionRpcAttachment,
ConnectionRpcFailure,
ConnectionRpcHandler,
ConnectionRpcHandlerResult,
ConnectionRequestRejection,
ConnectionRpcResult,
ConnectionRequestBodyMode,
+13 -55
View File
@@ -293,63 +293,21 @@ function errorResponse(rpcId: RpcIdType, error: ConnectionRpcFailure): Response
}
function fullResponse(rpcId: RpcIdType, result: Awaited<ReturnType<ConnectionRpcHandler>>): Response {
const body: ConnectionServerResponse = { type: 'server-response', rpcId, result }
if (!result.ok) return Response.json(body)
if (!result.ok) {
const body: ConnectionServerResponse = { type: 'server-response', rpcId, result }
return Response.json(body)
}
const { attachments, ...success } = result
const body: ConnectionServerResponse = { type: 'server-response', rpcId, result: success }
if (attachments === undefined || attachments.length === 0) return Response.json(body)
const parts = new FormData()
const attachments: { path: (string | number)[]; codec: 'bytes'; part: string }[] = []
const path: (string | number)[] = []
const ancestors = new Set<object>()
const writeBytes = (value: Uint8Array, path: readonly (string | number)[]): null => {
const part = `bytes-${attachments.length}`
attachments.push({ path: [...path], codec: 'bytes', part })
const attachmentMetadata = attachments.map((attachment, index) => {
const part = `bytes-${index}`
// FileSystem bytes may have SharedArrayBuffer backing, which BlobPart excludes.
parts.set(part, new Blob([new Uint8Array(value)]))
return null
}
const extract = (input: unknown, key: string): unknown => {
// Capture accessors and toJSON once, before materializing the JSON metadata.
let value = input
if (input !== null && typeof input === 'object' && !(input instanceof Uint8Array)) {
const toJSON: unknown = Reflect.get(input, 'toJSON')
if (typeof toJSON === 'function') value = Reflect.apply(toJSON, input, [key])
}
if (value instanceof Uint8Array) return writeBytes(value, path)
if (typeof value !== 'object' || value === null) return value
if (value instanceof Number || value instanceof String || value instanceof Boolean) return value
if (ancestors.has(value)) throw new TypeError('connection: circular RPC result')
ancestors.add(value)
let copy: object
if (Array.isArray(value)) {
const items: unknown[] = []
// JSON arrays include every index, even holes and non-enumerable elements.
for (let index = 0, length = value.length; index < length; index++) items.push(child(value[index], index))
copy = items
} else {
const fields: Record<string, unknown> = {}
for (const key of Object.keys(value)) {
const item: unknown = Reflect.get(value, key)
// A toJSON method on the projected object must not run a second time.
if (key === 'toJSON' && typeof item === 'function') continue
const extracted = child(item, key)
if (key === '__proto__') Object.defineProperty(fields, key, { value: extracted, enumerable: true })
else fields[key] = extracted
}
copy = fields
}
ancestors.delete(value)
return copy
}
const child = (value: unknown, key: string | number): unknown => {
if (typeof value !== 'object' || value === null) return value
path.push(key)
const extracted = extract(value, String(key))
path.pop()
return extracted
}
const value = result.encode === undefined ? extract(result.value, 'value') : result.encode(result.value, writeBytes)
const metadata = { ...body, result: { ok: true, value } }
if (attachments.length === 0) return Response.json(metadata)
parts.set('metadata', JSON.stringify({ ...metadata, attachments }))
parts.set(part, new Blob([new Uint8Array(attachment.bytes)]))
return { path: [...attachment.path], codec: 'bytes' as const, part }
})
parts.set('metadata', JSON.stringify({ ...body, attachments: attachmentMetadata }))
return new Response(parts)
}
@@ -6,7 +6,7 @@ import { describe, expect, it } from 'vitest'
import type { BrowserAuth } from '../src/browser-auth.ts'
import { createWebConnectionRpc } from '../src/client/rpc.ts'
import { bridge } from '../src/http-bridge.ts'
import type { ConnectionRpcResult } from '../src/rpc.ts'
import type { ConnectionRpcHandlerResult, ConnectionRpcResult } from '../src/rpc.ts'
import { HostConnectionService } from '../src/rpc-host.ts'
function binaryResponse(rpcId: string, mutate: (parts: FormData) => void = () => {}): Response {
@@ -29,7 +29,11 @@ describe('Connection binary RPC', () => {
const connection = ctx.get('connection') as HostConnectionService
const data = new Uint8Array(1024 * 1024).fill(65)
data.set([0, 128, 255])
connection.rpc.intercept('/api', endpoint => endpoint === 'fixture/read', async () => ({ ok: true, value: { data, bytes: data.length } }))
connection.rpc.intercept('/api', endpoint => endpoint === 'fixture/read', async () => ({
ok: true,
value: { data: null, bytes: data.length },
attachments: [{ path: ['data'], bytes: data }],
}))
const handler = connection.createSharedFetchHandler('/api')
ctx.webServer.register({ kind: 'prefix', path: '/api', handler: (req, res) => bridge(req, res, handler) })
let transferred = 0
@@ -73,17 +77,36 @@ describe('Connection binary RPC', () => {
await fiber.await()
try {
const connection = ctx.get('connection') as HostConnectionService
const results: ConnectionRpcResult<unknown>[] = [
{ ok: true, value: { data: new Uint8Array([9, 0, 128, 255, 9]).subarray(1, 4), offset: 7, eof: false, bytes: 42 } },
{ ok: true, value: { data: new Uint8Array(new SharedArrayBuffer(3)).fill(255), offset: 0, eof: true, bytes: 3 } },
{ ok: true, value: { data: new Uint8Array(), offset: 0, eof: true, bytes: 0 } },
{ ok: true, value: { data: 'AID/', encoding: 'base64' } },
{ ok: true, value: { count: 4 } },
{ ok: true, value: null },
{ ok: true, value: 'plain' },
{ ok: false, error: { code: 'fixture/denied', message: 'denied', details: { path: 'private' } } },
const byteValues = [
new Uint8Array([9, 0, 128, 255, 9]).subarray(1, 4),
new Uint8Array(new SharedArrayBuffer(3)).fill(255),
new Uint8Array(),
]
let next: ConnectionRpcResult<unknown> = results[0]!
const cases: readonly {
readonly sent: ConnectionRpcHandlerResult
readonly expected: ConnectionRpcResult<unknown>
}[] = [
...byteValues.map((data, index) => ({
sent: {
ok: true as const,
value: { data: null, offset: index === 0 ? 7 : 0, eof: index !== 0, bytes: index === 0 ? 42 : data.length },
attachments: [{ path: ['data'], bytes: data }],
},
expected: {
ok: true as const,
value: { data, offset: index === 0 ? 7 : 0, eof: index !== 0, bytes: index === 0 ? 42 : data.length },
},
})),
{ sent: { ok: true, value: { data: 'AID/', encoding: 'base64' } }, expected: { ok: true, value: { data: 'AID/', encoding: 'base64' } } },
{ sent: { ok: true, value: { count: 4 } }, expected: { ok: true, value: { count: 4 } } },
{ sent: { ok: true, value: null }, expected: { ok: true, value: null } },
{ sent: { ok: true, value: 'plain' }, expected: { ok: true, value: 'plain' } },
{
sent: { ok: false, error: { code: 'fixture/denied', message: 'denied', details: { path: 'private' } } },
expected: { ok: false, error: { code: 'fixture/denied', message: 'denied', details: { path: 'private' } } },
},
]
let next: ConnectionRpcHandlerResult = cases[0]!.sent
connection.rpc.intercept('/api', endpoint => endpoint === 'fixture/read', async () => next)
const shared = connection.createSharedFetchHandler('/api')
const mediaTypes: (string | null)[] = []
@@ -92,10 +115,10 @@ describe('Connection binary RPC', () => {
mediaTypes.push(response.headers.get('content-type'))
return response
})
for (const result of results) {
next = result
for (const { sent, expected } of cases) {
next = sent
const received = await rpc.call('/api', 'fixture/read', { args: {} })
expect(received).toEqual(result)
expect(received).toEqual(expected)
if (received.ok && typeof received.value === 'object' && received.value !== null
&& 'data' in received.value && received.value.data instanceof Uint8Array) {
expect(received.value.data.buffer).toBeInstanceOf(ArrayBuffer)
@@ -108,28 +131,23 @@ describe('Connection binary RPC', () => {
}
})
it('uses owner result codecs without putting their callbacks on the wire', async () => {
it('frames attachments already projected by the result owner', async () => {
const ctx = new Context()
const fiber = ctx.plugin((owner) => { new HostConnectionService(owner, [], {} as BrowserAuth) })
await fiber.await()
try {
const connection = ctx.get('connection') as HostConnectionService
const content = Buffer.from([9, 0, 128, 255, 9]).subarray(1, 4)
let reads = 0
const value = { content, metadata: { get count() { return ++reads } } }
connection.rpc.intercept('/api', () => true, async () => ({
ok: true, value,
encode: (input, writeBytes) => {
expect(input).toBe(value)
return { content: writeBytes(content, ['content']), metadata: value.metadata }
},
ok: true,
value: { content: null, metadata: { count: 1 } },
attachments: [{ path: ['content'], bytes: content }],
}))
const handler = connection.createSharedFetchHandler('/api')
const rpc = createWebConnectionRpc((url, init) => handler.fetch(new Request(new URL(url, 'http://host'), init)))
expect(await rpc.call('/api', 'fixture/read', {})).toEqual({
ok: true, value: { content: new Uint8Array([0, 128, 255]), metadata: { count: 1 } },
})
expect(reads).toBe(1)
} finally {
await fiber.dispose()
}
@@ -166,55 +184,56 @@ describe('Connection binary RPC', () => {
await expect(rpc.call('/api', 'fixture/read', {})).rejects.toThrow('rpcId mismatch')
})
it('roundtrips nested, optional and root bytes without changing caller objects or reserving field names', async () => {
it('roundtrips nested, optional and root attachment paths without reserving field names', async () => {
const ctx = new Context()
const fiber = ctx.plugin((owner) => { new HostConnectionService(owner, [], {} as BrowserAuth) })
await fiber.await()
try {
const data = new Uint8Array([0, 128, 255])
const shared = { content: data }
const value = {
files: [{ name: 'a.png', ...shared }, { name: 'b.png', content: Buffer.from([9, 2, 3, 9]).subarray(1, 3) }, { name: 'empty' }],
'dots./[]': [new Uint8Array(), null, [data]],
metadata: { attachments: shared },
...JSON.parse('{"__proto__":null,"constructor":null}') as object,
const second = Buffer.from([9, 2, 3, 9]).subarray(1, 3)
let next: ConnectionRpcHandlerResult = {
ok: true,
value: {
files: [{ name: 'a.png', content: null }, { name: 'b.png', content: null }, { name: 'empty' }],
'dots./[]': [null, null, [null]],
metadata: { attachments: { content: null } },
...JSON.parse('{"__proto__":null,"constructor":null}') as object,
},
attachments: [
{ path: ['files', 0, 'content'], bytes: data },
{ path: ['files', 1, 'content'], bytes: second },
{ path: ['dots./[]', 0], bytes: new Uint8Array() },
{ path: ['dots./[]', 2, 0], bytes: data },
{ path: ['metadata', 'attachments', 'content'], bytes: data },
{ path: ['__proto__'], bytes: data },
],
}
Object.defineProperty(value, '__proto__', { value: data, enumerable: true })
Object.freeze(value.files)
Object.freeze(value)
let next: unknown = value
const connection = ctx.get('connection') as HostConnectionService
connection.rpc.intercept('/api', () => true, async () => ({ ok: true, value: next }))
connection.rpc.intercept('/api', () => true, async () => next)
const handler = connection.createSharedFetchHandler('/api')
const rpc = createWebConnectionRpc((url, init) => handler.fetch(new Request(new URL(url, 'http://host'), init)))
expect(await rpc.call('/api', 'fixture/read', {})).toEqual({ ok: true, value: {
...value, files: [value.files[0], { name: 'b.png', content: new Uint8Array([2, 3]) }, value.files[2]],
} })
expect(value.files[0]?.content).toBe(data)
expect(shared.content).toBe(data)
expect(Object.getPrototypeOf(value)).toBe(Object.prototype)
const expected = {
files: [{ name: 'a.png', content: data }, { name: 'b.png', content: new Uint8Array([2, 3]) }, { name: 'empty' }],
'dots./[]': [new Uint8Array(), null, [data]],
metadata: { attachments: { content: data } },
...JSON.parse('{"__proto__":null,"constructor":null}') as object,
}
Object.defineProperty(expected, '__proto__', { value: data, enumerable: true })
expect(await rpc.call('/api', 'fixture/read', {})).toEqual({ ok: true, value: expected })
for (const bytes of [data, Buffer.from([0, 128, 255]), new Uint8Array(new SharedArrayBuffer(3))]) {
next = bytes
next = { ok: true, value: null, attachments: [{ path: [], bytes }] }
const response = await rpc.call('/api', 'fixture/read', {})
expect(response).toEqual({ ok: true, value: new Uint8Array(bytes) })
if (!response.ok || !(response.value instanceof Uint8Array)) throw new Error('expected root bytes')
expect(response.value.buffer).toBeInstanceOf(ArrayBuffer)
expect(Object.isFrozen(response.value)).toBe(false)
}
next = [data, [data]]
expect(await rpc.call('/api', 'fixture/read', {})).toEqual({ ok: true, value: next })
const circular: { next?: object } = {}
circular.next = circular
next = circular
await expect(rpc.call('/api', 'fixture/read', {})).rejects.toThrow('HTTP 500')
next = { ordinary: [1, 'base64', null], omitted: undefined, date: new Date('2026-01-01T00:00:00Z') }
expect(await rpc.call('/api', 'fixture/read', {})).toEqual({ ok: true, value: {
ordinary: [1, 'base64', null], date: '2026-01-01T00:00:00.000Z',
} })
const sparse = [data, , null]
Object.assign(sparse, { extra: data, '-1': data, '01': data, '1.5': data, Infinity: data })
next = sparse
expect(await rpc.call('/api', 'fixture/read', {})).toEqual({ ok: true, value: [data, null, null] })
next = {
ok: true,
value: [null, [null]],
attachments: [{ path: [0], bytes: data }, { path: [1, 0], bytes: data }],
}
expect(await rpc.call('/api', 'fixture/read', {})).toEqual({ ok: true, value: [data, [data]] })
} finally {
await fiber.dispose()
}
@@ -250,49 +269,6 @@ describe('Connection binary RPC', () => {
await expect(rpc.call('/api', 'fixture/read', {})).rejects.toThrow('invalid binary response')
})
it('preserves JSON accessor and toJSON evaluation while reading all array indices', async () => {
const ctx = new Context()
const fiber = ctx.plugin((owner) => { new HostConnectionService(owner, [], {} as BrowserAuth) })
await fiber.await()
try {
let reads = 0
let next: unknown = { get count() { return ++reads } }
const connection = ctx.get('connection') as HostConnectionService
connection.rpc.intercept('/api', () => true, async () => ({ ok: true, value: next }))
const handler = connection.createSharedFetchHandler('/api')
const rpc = createWebConnectionRpc((url, init) => handler.fetch(new Request(new URL(url, 'http://host'), init)))
const read = () => rpc.call('/api', 'fixture/read', {})
expect(await read()).toEqual({ ok: true, value: { count: 1 } })
expect(reads).toBe(1)
const bytes = new Uint8Array([1, 2])
next = { get content() { reads++; return bytes }, get count() { return ++reads } }
expect(await read()).toEqual({ ok: true, value: { content: bytes, count: 3 } })
const array: Uint8Array[] = []
Object.defineProperty(array, 0, { value: bytes })
next = array
expect(await read()).toEqual({ ok: true, value: [bytes] })
const date = Object.assign(new Date('2026-01-01T00:00:00Z'), { content: bytes })
next = date
expect(await read()).toEqual({ ok: true, value: '2026-01-01T00:00:00.000Z' })
const json = { self: {} as object, toJSON: () => ({ selected: true }) }
json.self = json
next = json
expect(await read()).toEqual({ ok: true, value: { selected: true } })
let conversions = 0
next = { toJSON(key: string) { conversions++; expect(key).toBe('value'); return this }, count: 5 }
expect(await read()).toEqual({ ok: true, value: { count: 5 } })
expect(conversions).toBe(1)
for (const primitive of [1, 'text', true]) {
next = Object(primitive) as object
expect(await read()).toEqual({ ok: true, value: primitive })
}
next = { toJSON: 'ordinary' }
expect(await read()).toEqual({ ok: true, value: next })
} finally {
await fiber.dispose()
}
})
it('rejects truncated multipart and late bytes after caller cancellation', async () => {
const broken = createWebConnectionRpc(async () => new Response('truncated', {
headers: { 'content-type': 'multipart/form-data; boundary=fixture' },
@@ -3365,22 +3365,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
returns: 'the page, the file\'s version at the stat before it, and whether it reaches the last line.',
},
{
signature: '@Remote async readBytes( workspaceFileScope: WorkspaceFileScope, path: string, range: WorkspaceByteRange, signal: AbortSignal, ): Promise<WorkspaceFileBytes>',
description: 'Read one byte window of a regular file readable by the filesystem backend: raw bytes, no text decoding and no binary rejection.',
parameters: [{ name: 'workspaceFileScope', description: 'header-derived workspace root for the Session identity on the wire.' }, { name: 'path', description: 'absolute path or path relative to the workspace root; files outside it are allowed.' }, { name: 'range', description: 'the byte window; omitted fields take the window defaults.' }, { name: 'signal', description: 'caller cancellation.' }],
returns: 'the window in base64, the file\'s version and size at the stat before it, and whether it reaches the last byte.',
},
{
signature: '@Remote async readAll(workspaceFileScope: WorkspaceFileScope, path: string, signal: AbortSignal): Promise<WorkspaceFileBytes>',
description: 'Read a complete regular file as bytes, subject to the configured full-file cap.',
parameters: [{ name: 'workspaceFileScope', description: 'header-derived workspace root for the Session identity on the wire.' }, { name: 'path', description: 'absolute or workspace-relative file path.' }, { name: 'signal', description: 'caller cancellation.' }],
returns: 'one complete base64 window with offset zero and eof true; oversized files fail with too-large.',
},
{
signature: '@Remote async readRelated( workspaceFileScope: WorkspaceFileScope, path: string, relativePath: string, signal: AbortSignal, ): Promise<WorkspaceFileBytes>',
description: 'Read a complete file relative to another file\'s directory, including outside the workspace.',
parameters: [{ name: 'workspaceFileScope', description: 'header-derived workspace root for the Session identity on the wire.' }, { name: 'path', description: 'base file, absolute or workspace-relative.' }, { name: 'relativePath', description: 'relative filesystem path, not a URL or absolute path.' }, { name: 'signal', description: 'caller cancellation.' }],
returns: 'the complete related file using the ordinary file-size and access checks.',
signature: '@Remote async readBytes( workspaceFileScope: WorkspaceFileScope, path: string, options: WorkspaceByteReadOptions, signal: AbortSignal, ): Promise<WorkspaceFileBytes>',
description: 'Read a complete regular file or one byte range without text decoding.',
parameters: [{ name: 'workspaceFileScope', description: 'header-derived workspace root for the Session identity on the wire.' }, { name: 'path', description: 'target path, absolute or workspace-relative; relative to the base file\'s directory when provided.' }, { name: 'options', description: 'optional base file and range; without a range the complete-file cap applies.' }, { name: 'signal', description: 'caller cancellation.' }],
returns: 'native bytes with the file\'s version and size at the preceding stat, byte offset, and EOF marker.',
},
{
signature: '@Remote async stat(workspaceFileScope: WorkspaceFileScope, path: string, signal: AbortSignal): Promise<WorkspaceFileStat>',
@@ -4452,6 +4440,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'ConnectionRequestRejection',
declaration: 'export type ConnectionRequestRejection = 401 | 403 | undefined;',
},
{
name: 'ConnectionRpcAttachment',
declaration: 'export interface ConnectionRpcAttachment {\n readonly path: readonly (string | number)[];\n readonly bytes: Uint8Array;\n}',
},
{
name: 'ConnectionRpcEndpointMatcher',
declaration: 'export type ConnectionRpcEndpointMatcher = (endpoint: string) => boolean;',
@@ -4462,7 +4454,11 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'ConnectionRpcHandler',
declaration: 'export type ConnectionRpcHandler = (endpoint: string, payload: unknown, signal: AbortSignal, peer: PeerScope) => Promise<ConnectionRpcResult<unknown>>;',
declaration: 'export type ConnectionRpcHandler = (endpoint: string, payload: unknown, signal: AbortSignal, peer: PeerScope) => Promise<ConnectionRpcHandlerResult>;',
},
{
name: 'ConnectionRpcHandlerResult',
declaration: 'export type ConnectionRpcHandlerResult = {\n readonly ok: true;\n readonly value: unknown;\n readonly attachments?: readonly ConnectionRpcAttachment[];\n} | {\n readonly ok: false;\n readonly error: ConnectionRpcFailure;\n};',
},
{
name: 'ConnectionRpcResult',
@@ -5702,7 +5698,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'RenderedDocumentBytes',
declaration: 'export interface RenderedDocumentBytes extends WorkspaceFileBytes {\n readonly missingFonts: string[];\n readonly generation: OfficeToPdfGeneration;\n}',
declaration: 'export interface RenderedDocumentBytes extends Omit<WorkspaceFileBytes, \'data\'> {\n readonly data: string;\n readonly missingFonts: string[];\n readonly generation: OfficeToPdfGeneration;\n}',
},
{
name: 'ReplayEnvelope',
@@ -7034,7 +7030,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'TypertCodec',
declaration: 'export type TypertCodec = {\n readonly mode: \'strict\';\n readonly typeSymbol: string;\n readonly create: () => TypertSchema;\n} | {\n readonly mode: \'src-json\';\n};',
declaration: 'export type TypertCodec = {\n readonly mode: \'strict\';\n readonly typeSymbol: string;\n readonly create: () => TypertSchema;\n readonly decode?: (value: unknown) => unknown;\n readonly encode?: (value: unknown, writeBytes: (bytes: Uint8Array, path: readonly (string | number)[]) => null) => unknown;\n} | {\n readonly mode: \'src-json\';\n};',
},
{
name: 'TypertContribution',
@@ -7332,6 +7328,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'WorkspaceByteRange',
declaration: 'export interface WorkspaceByteRange {\n readonly offset?: number;\n readonly length?: number;\n}',
},
{
name: 'WorkspaceByteReadOptions',
declaration: 'export interface WorkspaceByteReadOptions {\n readonly range?: WorkspaceByteRange;\n readonly baseFile?: string;\n}',
},
{
name: 'WorkspaceChangedFile',
declaration: 'export interface WorkspaceChangedFile {\n path: string;\n display: string;\n added: number;\n deleted: number;\n binary?: true;\n oversized?: true;\n}',
@@ -7370,7 +7370,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'WorkspaceFileBytes',
declaration: 'export interface WorkspaceFileBytes extends WorkspaceFileStat {\n readonly offset: number;\n readonly data: string;\n readonly eof: boolean;\n}',
declaration: 'export interface WorkspaceFileBytes<Data extends Uint8Array = Uint8Array> extends WorkspaceFileStat {\n readonly offset: number;\n readonly data: Data;\n readonly eof: boolean;\n}',
},
{
name: 'WorkspaceFileChange',
+1 -1
View File
@@ -280,7 +280,7 @@ export type TypertCodec =
*/
readonly decode?: (value: unknown) => unknown
/**
* Project typed binary fields into carrier-owned multipart attachments.
* Project typed binary fields into RPC result attachments.
* @param value - native unary result.
* @param writeBytes - records a byte view at its result-relative path and returns its JSON placeholder.
* @returns JSON metadata with untouched JSON subtrees retained.