fix(web): display local media paths referenced in session prose
Assistant prose that references a workspace-contained local media path (e.g. ``) now renders through a same-origin `GET|HEAD /api/file?path=` route instead of inert alt text. - ui-primitives: MarkdownText gains a settled-only MarkdownPathImages vocabulary gate (same posture as file mentions); no vocabulary means byte-identical output. - ui-chat: AssistantMarkdown supplies a page-stable rewrite vocabulary for absolute POSIX paths (local-path-media.ts). - session-controller: SessionMediaReferences plugin contribution mounts the route on the authenticated connection.fetch channel; per-request policy = workspace-root containment after realpath, regular file, allowlisted media extension (images additionally signature-checked), range/HEAD streaming, private no-store + nosniff, fail-closed statuses. - Agent Note added (feature/2026-09-07-session-prose-local-media-display). Closes #3662.
This commit is contained in:
1 parent
5608528e14
commit
9da174c304
16 files changed
+872
-13
No files matched your search
+6
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.md
|
||||
2026-09-07-session-prose-local-media-display.md: e2693a6192cb78641d8a96937b39a4b06e884f04
|
||||
2026-09-07-session-prose-local-media-display.zh.md: bc0c62868e3ffab44fc3effc1030d9caee331f26
|
||||
@@ -0,0 +1,39 @@
|
||||
# Agent Note: Session prose local media paths display through a same-origin file route
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-09-07-session-prose-local-media-display.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Assistant prose sometimes references an image by its local filesystem path (markdown ``). The Web renderer only allowed absolute HTTP(S) image destinations, so such references fell back to inert alt text: the browser cannot read Host files, and nothing served them. Searches of the formal and external issue trackers found no existing record, and issue #3662 logged the gap (Web cannot display local-path images referenced by agent answers).
|
||||
|
||||
## Decision
|
||||
|
||||
Local media paths in session prose render through one same-origin file route, with the rewrite vocabulary and the serving policy each owned where the repo's seams say they belong.
|
||||
|
||||
- **Renderer seam (`ui-primitives`)**: `MarkdownText` gained a `MarkdownPathImages` vocabulary (`pathImages` prop) with the same settled-only gate as `fileMentions`: while a message streams, frozen cached blocks never bake in a vocabulary handler; the settled pass rewrites image destinations that fail the remote-URL allowlist, and a rewritten destination is emitted only when it is an absolute `http(s)`/`blob`/`data` URL. Without a vocabulary the renderer output is byte-identical to before.
|
||||
- **Chat wiring (`ui-chat`)**: `AssistantMarkdown` supplies a page-stable vocabulary mapping absolute POSIX paths to same-origin `/api/file?path=…` GETs (`local-path-media.ts`); non-HTTP transports (Electron `file://`) and relative/protocol-relative destinations stay inert.
|
||||
- **Host route (`session-controller`)**: `SessionMediaReferences`, a plugin contribution registered beside `SessionFileReferences`, mounts `GET|HEAD /api/file` on the shared authenticated `connection.fetch` channel (same trust fence and browser authentication as `/api` RPC). Per request it fail-closes: the path must be absolute, its `realpath` must lie inside a registered workspace root, the file must be regular, and its extension must name an allowlisted media type (PNG/JPEG/GIF/WebP/AVIF, MP4/WebM/MOV/OGG, MP3/WAV/Ogg/M4A/AAC/FLAC); image extensions are additionally checked against their byte signature. Responses stream with HTTP range support (206/416) so video and audio can seek, and carry `private, no-store` and `nosniff`. The contribution activates only where `connection` and `workspaceRegistry` are composed (pending-until-composed, like the package's other optional contributions).
|
||||
|
||||
The route is presentational and stateless: it never writes, follows no redirects, and returns 400/403/404/415/416 instead of approximating another file-serving behavior.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Register on the Typert gateway**: the gateway owns Remote RPC dispatch (endpoint claims, WebSocket mux, forwarded events), not file serving; placing the route there put HTTP presentation of workspace files into the RPC transport layer. Rejected and fully reverted.
|
||||
- **Register under `workspace-controller`**: that package owns workspace registry lifecycle (CRUD, ordering, feed), and shares only the registry as a policy data source. Rejected and fully reverted.
|
||||
- **Fetch bytes over the session RPC and show blob/data URLs**: attachment images already do this, but markdown rewriting needs a deterministic synchronous URL at render time (streaming freeze caches, memoization); an async round trip cannot be the render seam. Rejected.
|
||||
- **A per-image or image-only route**: media types share one path-and-contain policy; video/audio need range streaming anyway. One `/api/file` route with an extension allowlist plus signature checks for images covers all current and next media types. Rejected as narrower alternatives.
|
||||
- **Per-request interactive authorization, client-negotiated endpoints, or arbitrary host paths**: the rewrite is presentation; the route re-validates every request and limits readable bytes to registered workspace roots and allowlisted media, and the same-origin endpoint is a fixed channel contract rather than a negotiated capability. Rejected for security and determinism reasons.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Assistant prose that references workspace-contained image files now displays them in Web chat; previously inert alt text disappears only where the host can serve the bytes.
|
||||
- Policy is enforced host-side per request; the client vocabulary never expands what the route allows.
|
||||
- Scope is deliberately narrow: only files under registered workspace roots with allowlisted media types are served; anything else keeps the authored fallback. Trajectory and tool-card markdown consumers do not pass a vocabulary yet, and video/audio markdown nodes are not rendered as `<video>`/`<audio>` yet — the URL layer already supports them.
|
||||
- The client hardcodes the `/api/file` endpoint as a same-origin channel contract; this is stable by construction because the page is served by the same host that mounts the route.
|
||||
- Related history: the archived note [model-readable image paths](../../archived/feature/2026-08-21-model-readable-image-paths.md) decided the model-facing side of local image paths; this note owns the user-facing display side and does not supersede it.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit coverage: the renderer seam (settled and streaming gates, reference-style images, protocol re-checks), the chat vocabulary and component wiring, and the host route policy (containment, symlinks, media allowlist and signature mismatch, range parsing and 206/416 answers, HEAD, disposal of the plugin contribution). Full local evidence at open: typecheck clean; ui-primitives, ui-chat, and session-controller suites (103 files, 1456 tests) pass. Deferred to the PR follow-up: keyless recorded-session replay for the end-to-end GUI path and the browser demo GIF the GUI PR evidence chain requires.
|
||||
@@ -0,0 +1,39 @@
|
||||
# Agent Note: 会话正文本地媒体路径通过同源文件路由显示
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-09-07-session-prose-local-media-display.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
Assistant 正文有时用本地文件系统路径引用图片(markdown ``)。Web 渲染器只允许绝对 HTTP(S) 图片目标,这类引用只能回退为惰性 alt 文本:浏览器读不到 Host 文件,也没有任何东西提供这些字节。检索正式与外部 Issue 仓库均无既有记录,issue #3662 记录了这一缺口(Web 无法显示 Agent 回答引用的本地路径图片)。
|
||||
|
||||
## Decision
|
||||
|
||||
会话正文中的本地媒体路径通过一条同源文件路由渲染;重写词表与服务政策分别归属于仓库现有缝对应的位置。
|
||||
|
||||
- **渲染缝(`ui-primitives`)**:`MarkdownText` 新增 `MarkdownPathImages` 词表(`pathImages` prop),与 `fileMentions` 同用 settled-only 门:消息流式期间冻结的缓存块绝不烘入词表处理器;落定渲染把未通过远程 URL 白名单的图片目标重写为可展示 URL,且只有结果是绝对 `http(s)`/`blob`/`data` URL 时才发射。不提供词表时渲染输出与之前逐字节一致。
|
||||
- **聊天接线(`ui-chat`)**:`AssistantMarkdown` 提供页面级稳定的词表,把绝对 POSIX 路径映射为同源 `GET /api/file?path=…`(`local-path-media.ts`);非 HTTP 载体(Electron `file://`)与相对/协议相对目标保持惰性。
|
||||
- **Host 路由(`session-controller`)**:`SessionMediaReferences` 是与 `SessionFileReferences` 并列注册的插件贡献,把 `GET|HEAD /api/file` 挂到共享鉴权 `connection.fetch` 通道(与 `/api` RPC 同一 trust fence 与浏览器认证)。每次请求 fail-closed:路径必须绝对、其 `realpath` 必须落在已注册 workspace 根内、文件必须是常规文件、扩展名必须在媒体 allowlist 内(PNG/JPEG/GIF/WebP/AVIF、MP4/WebM/MOV/OGG、MP3/WAV/Ogg/M4A/AAC/FLAC);图片扩展额外校验字节签名。响应以 HTTP Range 流式输出(206/416),视频/音频可拖动进度,并携带 `private, no-store` 与 `nosniff`。该贡献只在 `connection` 与 `workspaceRegistry` 均被组合时激活(pending-until-composed,与包内其它可选贡献一致)。
|
||||
|
||||
该路由纯呈现且无状态:从不写入、不跟随重定向,失败返回 400/403/404/415/416,不近似其它文件服务行为。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **注册到 Typert gateway**:gateway 负责 Remote RPC 分发(endpoint 认领、WebSocket mux、转发事件),不负责文件服务;把路由放那里等于把 workspace 文件的 HTTP 呈现塞进 RPC 传输层。否决并完整回滚。
|
||||
- **注册到 `workspace-controller`**:该包负责 workspace 注册表生命周期(CRUD、排序、feed),与文件呈现只共享 registry 这一政策数据源。否决并完整回滚。
|
||||
- **经会话 RPC 取字节后显示 blob/data URL**:附件图片已如此工作,但 markdown 重写需要渲染时确定性同步 URL(流式冻结缓存、memo 化);异步往返不能成为渲染缝。否决。
|
||||
- **逐图片或仅图片专用路由**:媒体类型共享同一条「路径 + 包含」政策,视频/音频本就需要 Range 流式;一条 `/api/file` 路由加扩展名 allowlist(图片另验签名)即可覆盖现有与后续媒体类型。作为更窄的方案被否决。
|
||||
- **每请求交互授权、客户端协商端点或任意 Host 路径**:重写只是呈现;路由对每次请求复验,可读字节限制在注册 workspace 根与 allowlist 媒体内;同源端点是固定通道契约而非协商能力。出于安全与确定性否决。
|
||||
|
||||
## Consequences
|
||||
|
||||
- 引用 workspace 内图片文件的 Assistant 正文现在可在 Web 聊天中显示;原先惰性的 alt 文本只在 Host 无法提供字节时保留。
|
||||
- 政策在 Host 端每次请求强制;客户端词表不会扩大路由放行的范围。
|
||||
- 范围刻意收窄:只服务注册 workspace 根内、媒体 allowlist 内的文件;其它一律保持作者原样的回退。Trajectory 与工具卡片等 markdown 消费方尚未传词表,视频/音频 markdown 节点也尚未渲染为 `<video>`/`<audio>`——URL 层已为它们准备好。
|
||||
- 客户端把 `/api/file` 硬编码为同源通道契约;由于页面与挂载路由的 Host 同源,该契约按构造稳定。
|
||||
- 相关历史:已归档笔记 [model-readable image paths](../../archived/feature/2026-08-21-model-readable-image-paths.md) 决定本地图片路径面向模型的一侧;本笔记拥有面向用户展示的一侧,不构成对其的取代。
|
||||
|
||||
## Testing
|
||||
|
||||
单元覆盖:渲染缝(落定与流式门、引用式图片、协议复检)、聊天词表与组件接线、Host 路由政策(包含关系、符号链接、媒体 allowlist 与签名不符、Range 解析与 206/416 应答、HEAD、插件贡献的释放)。开 PR 时的本地证据:typecheck 干净;ui-primitives、ui-chat 与 session-controller 套件(103 个文件、1456 个用例)全过。推迟到 PR 后续:端到端 GUI 路径的 keyless 录播回放,以及 GUI PR 证据链要求的浏览器演示 GIF。
|
||||
@@ -22,6 +22,7 @@ import { ApiSessionList } from './list.ts'
|
||||
import { buildModelCatalog } from './catalog.ts'
|
||||
import { installModelSelectionProjection } from './model-selection-projection.ts'
|
||||
import { SessionSkillCatalog } from './skill-catalog.ts'
|
||||
import { SessionMediaReferences } from './media-references.ts'
|
||||
import type {
|
||||
ModelCatalog,
|
||||
SessionAttachmentRequest,
|
||||
@@ -134,6 +135,7 @@ export class SessionController extends TypertRemoteService {
|
||||
this.canOpenPath = internals.canOpenPath
|
||||
?? (() => config.nativeOpen ?? (internals.openPath !== undefined || canOpenNativePath()))
|
||||
ctx.plugin(SessionFileReferences)
|
||||
ctx.plugin(SessionMediaReferences)
|
||||
ctx.plugin(SessionSkillCatalog)
|
||||
|
||||
ctx.on('session/created', (session) => {
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
/**
|
||||
* GET/HEAD /api/file — same-origin media bytes for a local path authored in
|
||||
* conversation prose. Browsers cannot read Host files, so local media
|
||||
* destinations (images today; video/audio the same way) are rewritten to this
|
||||
* one route; the route re-validates every request because the rewrite itself
|
||||
* proves nothing.
|
||||
*
|
||||
* Policy (enforced per request, fail-closed):
|
||||
* - the requested path must be one absolute filesystem path;
|
||||
* - its canonical location must lie inside a registered workspace root
|
||||
* (`ctx.workspaceRegistry`); no other directory is readable;
|
||||
* - the file must exist and be a regular file;
|
||||
* - its extension must name an allowlisted media type; image extensions are
|
||||
* additionally checked against the file signature;
|
||||
* - responses are private, uncached, sniff-proof, and support HTTP range
|
||||
* requests so `<video>`/`<audio>` can seek without buffering the file.
|
||||
*
|
||||
* The route is deliberately presentational: it never writes and follows no
|
||||
* redirects, returning 400/403/404/415/416 instead of falling back to any
|
||||
* other file-serving behavior.
|
||||
*
|
||||
* The package entry imports only `SessionMediaReferences`; the remaining
|
||||
* module exports exist for same-package unit tests and are not part of the
|
||||
* package's public API.
|
||||
* @module @deepseek-ai/dsh-api-session-controller/media-references
|
||||
*/
|
||||
|
||||
import { createReadStream } from 'node:fs'
|
||||
import { open, realpath, stat } from 'node:fs/promises'
|
||||
import { extname, isAbsolute, sep } from 'node:path'
|
||||
import { Readable } from 'node:stream'
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
// Cordis `ctx.connection` typing and the fetch-route contract.
|
||||
import type {} from '@deepseek-ai/dsh-client-connection'
|
||||
// Cordis `ctx.workspaceRegistry` typing.
|
||||
import type {} from '@deepseek-ai/dsh-workspace'
|
||||
|
||||
/** Registered-workspace view the route reads; see the `workspaceRegistry` service. */
|
||||
export interface MediaReferenceRegistry {
|
||||
/** List registered workspaces with their canonical root directories. */
|
||||
list(): readonly { path: string }[]
|
||||
}
|
||||
|
||||
/** Media extensions the route serves, mapped to their content types. */
|
||||
const MEDIA_TYPES: Readonly<Record<string, string>> = {
|
||||
'.png': 'image/png',
|
||||
'.jpg': 'image/jpeg',
|
||||
'.jpeg': 'image/jpeg',
|
||||
'.gif': 'image/gif',
|
||||
'.webp': 'image/webp',
|
||||
'.avif': 'image/avif',
|
||||
'.mp4': 'video/mp4',
|
||||
'.webm': 'video/webm',
|
||||
'.mov': 'video/quicktime',
|
||||
'.ogv': 'video/ogg',
|
||||
'.mp3': 'audio/mpeg',
|
||||
'.wav': 'audio/wav',
|
||||
'.ogg': 'audio/ogg',
|
||||
'.oga': 'audio/ogg',
|
||||
'.m4a': 'audio/mp4',
|
||||
'.aac': 'audio/aac',
|
||||
'.flac': 'audio/flac',
|
||||
}
|
||||
|
||||
/** Image extensions whose bytes must match their declared signature. */
|
||||
const SNIFFED_IMAGE_TYPES: Readonly<Record<string, string>> = {
|
||||
'.png': 'image/png',
|
||||
'.jpg': 'image/jpeg',
|
||||
'.jpeg': 'image/jpeg',
|
||||
'.gif': 'image/gif',
|
||||
'.webp': 'image/webp',
|
||||
}
|
||||
|
||||
const PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a] as const
|
||||
const JPEG_SIGNATURE = [0xff, 0xd8, 0xff] as const
|
||||
const GIF_87_SIGNATURE = 'GIF87a' as const
|
||||
const GIF_89_SIGNATURE = 'GIF89a' as const
|
||||
const RIFF_SIGNATURE = 'RIFF' as const
|
||||
const WEBP_SIGNATURE = 'WEBP' as const
|
||||
|
||||
function startsWithBytes(data: Uint8Array, offset: number, expected: readonly number[]): boolean {
|
||||
if (data.byteLength < offset + expected.length) return false
|
||||
return expected.every((byte, index) => data[offset + index] === byte)
|
||||
}
|
||||
|
||||
function startsWithAscii(data: Uint8Array, offset: number, value: string): boolean {
|
||||
if (data.byteLength < offset + value.length) return false
|
||||
for (let index = 0; index < value.length; index += 1) {
|
||||
if (data[offset + index] !== value.charCodeAt(index)) return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
/**
|
||||
* Check one byte buffer against the sniffed image signatures.
|
||||
* @param data - Leading file bytes (12 suffice for every supported type).
|
||||
* @returns the matched media type, or undefined for other bytes.
|
||||
*/
|
||||
export function sniffImageMediaType(data: Uint8Array): string | undefined {
|
||||
if (startsWithBytes(data, 0, PNG_SIGNATURE)) return 'image/png'
|
||||
if (startsWithBytes(data, 0, JPEG_SIGNATURE)) return 'image/jpeg'
|
||||
if (startsWithAscii(data, 0, GIF_87_SIGNATURE) || startsWithAscii(data, 0, GIF_89_SIGNATURE)) {
|
||||
return 'image/gif'
|
||||
}
|
||||
if (startsWithAscii(data, 0, RIFF_SIGNATURE) && startsWithAscii(data, 8, WEBP_SIGNATURE)) {
|
||||
return 'image/webp'
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* The content type a file path may be served as, or undefined when the
|
||||
* extension is not an allowlisted media type.
|
||||
* @param path - Canonical file path (extension only is read).
|
||||
* @param bytes - Leading file bytes for signature-checked image extensions.
|
||||
* @returns the content type, or undefined to refuse the file.
|
||||
*/
|
||||
export function mediaTypeForPath(path: string, bytes: Uint8Array): string | undefined {
|
||||
const extension = extname(path).toLowerCase()
|
||||
const sniffed = SNIFFED_IMAGE_TYPES[extension]
|
||||
if (sniffed !== undefined) return sniffImageMediaType(bytes) === sniffed ? sniffed : undefined
|
||||
return MEDIA_TYPES[extension]
|
||||
}
|
||||
|
||||
/** One resolved byte-range within a file; `full` streams the whole file. */
|
||||
type ByteRange =
|
||||
| { readonly kind: 'full' }
|
||||
| { readonly kind: 'partial'; readonly start: number; readonly end: number }
|
||||
|
||||
/**
|
||||
* Parse one single-range `Range` header value against the file size.
|
||||
* @param header - Raw `Range` request header, or null when absent.
|
||||
* @param size - Total file size in bytes.
|
||||
* @returns the byte range to serve, or undefined when the header names no
|
||||
* satisfiable single range (the caller answers 416 with a `bytes *\/size`
|
||||
* Content-Range header).
|
||||
*/
|
||||
export function parseByteRange(header: string | null, size: number): ByteRange | undefined {
|
||||
if (header === null) return { kind: 'full' }
|
||||
const match = /^bytes=(\d*)-(\d*)$/.exec(header.trim())
|
||||
if (match === null) return undefined
|
||||
const startText = match[1]
|
||||
const endText = match[2]
|
||||
if (startText === '' && endText === '') return undefined
|
||||
let start: number
|
||||
let end: number
|
||||
if (startText === '') {
|
||||
// Suffix range: the last N bytes.
|
||||
const length = Number(endText)
|
||||
if (length === 0) return undefined
|
||||
start = Math.max(size - length, 0)
|
||||
end = size - 1
|
||||
} else {
|
||||
start = Number(startText)
|
||||
end = endText === '' ? size - 1 : Number(endText)
|
||||
}
|
||||
if (start > end || start >= size) return undefined
|
||||
return { kind: 'partial', start, end: Math.min(end, size - 1) }
|
||||
}
|
||||
|
||||
/**
|
||||
* Serve one workspace-contained media file over the shared API channel.
|
||||
* @param request - Authenticated fetch-route request (GET or HEAD).
|
||||
* @param registry - Workspace registry; absent (or empty) denies everything.
|
||||
* @returns A streaming media response or a fail-closed status.
|
||||
*/
|
||||
export async function serveMediaReference(
|
||||
request: Request,
|
||||
registry: MediaReferenceRegistry | undefined,
|
||||
): Promise<Response> {
|
||||
const path = new URL(request.url).searchParams.get('path')
|
||||
if (path === null || path.length === 0) return new Response('missing path', { status: 400 })
|
||||
if (path.includes('\0') || !isAbsolute(path)) {
|
||||
return new Response('absolute path required', { status: 400 })
|
||||
}
|
||||
if (registry === undefined) return new Response('file serving is unavailable', { status: 403 })
|
||||
let canonical: string
|
||||
try {
|
||||
canonical = await realpath(path)
|
||||
} catch {
|
||||
return new Response('not found', { status: 404 })
|
||||
}
|
||||
const insideWorkspace = registry.list().some(root =>
|
||||
canonical === root.path || canonical.startsWith(root.path + sep))
|
||||
if (!insideWorkspace) return new Response('outside workspace roots', { status: 403 })
|
||||
let info
|
||||
try {
|
||||
info = await stat(canonical)
|
||||
} catch {
|
||||
return new Response('not found', { status: 404 })
|
||||
}
|
||||
if (!info.isFile()) return new Response('not a regular file', { status: 403 })
|
||||
let handle
|
||||
try {
|
||||
handle = await open(canonical, 'r')
|
||||
} catch {
|
||||
return new Response('not found', { status: 404 })
|
||||
}
|
||||
try {
|
||||
const headBytes = new Uint8Array(12)
|
||||
const { buffer } = await handle.read(headBytes, 0, 12, 0)
|
||||
const mediaType = mediaTypeForPath(canonical, new Uint8Array(buffer))
|
||||
if (mediaType === undefined) {
|
||||
return new Response('not an allowlisted media type', { status: 415 })
|
||||
}
|
||||
const range = parseByteRange(request.headers.get('range'), info.size)
|
||||
if (range === undefined) {
|
||||
const headers: Record<string, string> = {
|
||||
'Content-Range': 'bytes */' + String(info.size),
|
||||
'Cache-Control': 'private, no-store',
|
||||
'X-Content-Type-Options': 'nosniff',
|
||||
}
|
||||
return new Response(null, { status: 416, headers })
|
||||
}
|
||||
const partial = range.kind === 'partial'
|
||||
const start = partial ? range.start : 0
|
||||
const end = partial ? range.end : info.size - 1
|
||||
const body: ReadableStream<Uint8Array> = Readable.toWeb(
|
||||
createReadStream(canonical, { start, end }),
|
||||
) as ReadableStream<Uint8Array>
|
||||
const headers: Record<string, string> = {
|
||||
'Content-Type': mediaType,
|
||||
'Content-Length': String(partial ? end - start + 1 : info.size),
|
||||
'Accept-Ranges': 'bytes',
|
||||
'Cache-Control': 'private, no-store',
|
||||
'X-Content-Type-Options': 'nosniff',
|
||||
}
|
||||
if (partial) headers['Content-Range'] = 'bytes ' + start + '-' + end + '/' + String(info.size)
|
||||
const head = request.method === 'HEAD'
|
||||
return new Response(head ? null : body, { status: partial ? 206 : 200, headers })
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* `/api/file` fetch-route contribution, the media counterpart of
|
||||
* `SessionFileReferences`: that contribution lets the GUI discover the files
|
||||
* a Session references, this one lets it display referenced local media
|
||||
* paths as same-origin bytes. Declared injects gate activation: a host
|
||||
* composition without a connection service (or a workspace registry) never
|
||||
* activates this plugin, so the route simply does not exist there — the same
|
||||
* pending-until-composed posture the package's other optional contributions
|
||||
* use. The channel's trust fence and browser authentication apply before any
|
||||
* request reaches the handler.
|
||||
*/
|
||||
export const SessionMediaReferences = {
|
||||
inject: ['connection', 'workspaceRegistry'],
|
||||
apply(ctx: Context): void {
|
||||
ctx.effect(() => ctx.connection.fetch.register({
|
||||
path: '/api/file',
|
||||
methods: ['GET', 'HEAD'],
|
||||
requestBody: 'buffered',
|
||||
fetch: request => serveMediaReference(request, ctx.workspaceRegistry),
|
||||
}), 'session-controller: /api/file')
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,265 @@
|
||||
import { mkdir, mkdtemp, open, realpath, rm, symlink, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import {
|
||||
SessionMediaReferences,
|
||||
serveMediaReference,
|
||||
mediaTypeForPath,
|
||||
parseByteRange,
|
||||
sniffImageMediaType,
|
||||
type MediaReferenceRegistry,
|
||||
} from '../src/media-references.ts'
|
||||
|
||||
const PNG_BYTES = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 1, 2, 3, 4, 5, 6])
|
||||
const TEXT_BYTES = new Uint8Array([1, 2, 3, 4])
|
||||
const MP4_BYTES = new TextEncoder().encode('....ftypmp42....moov....')
|
||||
|
||||
function registry(root: string): MediaReferenceRegistry {
|
||||
return { list: () => [{ path: root }] }
|
||||
}
|
||||
|
||||
async function responseBytes(response: Response): Promise<Uint8Array> {
|
||||
return new Uint8Array(await response.arrayBuffer())
|
||||
}
|
||||
|
||||
function apiRequest(path: string, init?: RequestInit): Request {
|
||||
const url = `http://127.0.0.1:3080/api/file?path=${encodeURIComponent(path)}`
|
||||
return new Request(url, init)
|
||||
}
|
||||
|
||||
describe('sniffImageMediaType and mediaTypeForPath', () => {
|
||||
it('identifies supported image signatures', () => {
|
||||
expect(sniffImageMediaType(PNG_BYTES)).toBe('image/png')
|
||||
expect(sniffImageMediaType(new Uint8Array([0xff, 0xd8, 0xff, 0xe0]))).toBe('image/jpeg')
|
||||
expect(sniffImageMediaType(new TextEncoder().encode('GIF89a...'))).toBe('image/gif')
|
||||
expect(sniffImageMediaType(new TextEncoder().encode('RIFF0000WEBPVP8 '))).toBe('image/webp')
|
||||
expect(sniffImageMediaType(TEXT_BYTES)).toBeUndefined()
|
||||
})
|
||||
|
||||
it('maps allowlisted media extensions, sniffing image extensions', () => {
|
||||
expect(mediaTypeForPath('/w/graph.png', PNG_BYTES)).toBe('image/png')
|
||||
expect(mediaTypeForPath('/w/graph.jpg', new Uint8Array([0xff, 0xd8, 0xff]))).toBe('image/jpeg')
|
||||
expect(mediaTypeForPath('/w/clip.mp4', MP4_BYTES)).toBe('video/mp4')
|
||||
expect(mediaTypeForPath('/w/clip.webm', MP4_BYTES)).toBe('video/webm')
|
||||
expect(mediaTypeForPath('/w/song.mp3', TEXT_BYTES)).toBe('audio/mpeg')
|
||||
})
|
||||
|
||||
it('refuses non-media extensions and mislabeled image bytes', () => {
|
||||
expect(mediaTypeForPath('/w/note.txt', TEXT_BYTES)).toBeUndefined()
|
||||
expect(mediaTypeForPath('/w/app.exe', TEXT_BYTES)).toBeUndefined()
|
||||
expect(mediaTypeForPath('/w/shell.svg', TEXT_BYTES)).toBeUndefined()
|
||||
expect(mediaTypeForPath('/w/fake.png', TEXT_BYTES)).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
describe('parseByteRange', () => {
|
||||
it('parses full, bounded, open-ended, and suffix ranges', () => {
|
||||
expect(parseByteRange(null, 100)).toEqual({ kind: 'full' })
|
||||
expect(parseByteRange('bytes=0-3', 100)).toEqual({ kind: 'partial', start: 0, end: 3 })
|
||||
expect(parseByteRange('bytes=10-', 100)).toEqual({ kind: 'partial', start: 10, end: 99 })
|
||||
expect(parseByteRange('bytes=-4', 100)).toEqual({ kind: 'partial', start: 96, end: 99 })
|
||||
expect(parseByteRange('bytes=90-200', 100)).toEqual({ kind: 'partial', start: 90, end: 99 })
|
||||
})
|
||||
|
||||
it('refuses malformed and unsatisfiable ranges', () => {
|
||||
expect(parseByteRange('bytes=abc', 100)).toBeUndefined()
|
||||
expect(parseByteRange('bytes=5-2', 100)).toBeUndefined()
|
||||
expect(parseByteRange('bytes=100-', 100)).toBeUndefined()
|
||||
expect(parseByteRange('bytes=0-', 0)).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
describe('serveMediaReference', () => {
|
||||
let root: string
|
||||
|
||||
beforeEach(async () => {
|
||||
// Real workspace roots are canonical; tmpdir may sit behind a symlink.
|
||||
root = await realpath(await mkdtemp(join(tmpdir(), 'dsh-workspace-file-')))
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
await rm(root, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
it('serves a workspace image with streaming-safe headers', async () => {
|
||||
const path = join(root, 'shots', 'graph.png')
|
||||
await mkdir(join(root, 'shots'), { recursive: true })
|
||||
await writeFile(path, PNG_BYTES)
|
||||
const response = await serveMediaReference(apiRequest(path), registry(root))
|
||||
expect(response.status).toBe(200)
|
||||
expect(response.headers.get('content-type')).toBe('image/png')
|
||||
expect(response.headers.get('content-length')).toBe(String(PNG_BYTES.length))
|
||||
expect(response.headers.get('accept-ranges')).toBe('bytes')
|
||||
expect(response.headers.get('cache-control')).toBe('private, no-store')
|
||||
expect(response.headers.get('x-content-type-options')).toBe('nosniff')
|
||||
expect(await responseBytes(response)).toEqual(PNG_BYTES)
|
||||
})
|
||||
|
||||
it('serves media without image sniffing from the extension allowlist', async () => {
|
||||
const path = join(root, 'clip.mp4')
|
||||
await writeFile(path, MP4_BYTES)
|
||||
const response = await serveMediaReference(apiRequest(path), registry(root))
|
||||
expect(response.status).toBe(200)
|
||||
expect(response.headers.get('content-type')).toBe('video/mp4')
|
||||
expect(await responseBytes(response)).toEqual(MP4_BYTES)
|
||||
})
|
||||
|
||||
it('answers bounded and suffix range requests with 206 slices', async () => {
|
||||
const path = join(root, 'graph.png')
|
||||
await writeFile(path, PNG_BYTES)
|
||||
const bounded = await serveMediaReference(
|
||||
apiRequest(path, { headers: { range: 'bytes=0-3' } }),
|
||||
registry(root),
|
||||
)
|
||||
expect(bounded.status).toBe(206)
|
||||
expect(bounded.headers.get('content-range')).toBe(`bytes 0-3/${PNG_BYTES.length}`)
|
||||
expect(bounded.headers.get('content-length')).toBe('4')
|
||||
expect(await responseBytes(bounded)).toEqual(PNG_BYTES.slice(0, 4))
|
||||
|
||||
const suffix = await serveMediaReference(
|
||||
apiRequest(path, { headers: { range: 'bytes=-4' } }),
|
||||
registry(root),
|
||||
)
|
||||
expect(suffix.status).toBe(206)
|
||||
expect(await responseBytes(suffix)).toEqual(PNG_BYTES.slice(-4))
|
||||
})
|
||||
|
||||
it('answers unsatisfiable ranges with 416 and the total size', async () => {
|
||||
const path = join(root, 'graph.png')
|
||||
await writeFile(path, PNG_BYTES)
|
||||
const response = await serveMediaReference(
|
||||
apiRequest(path, { headers: { range: 'bytes=999-' } }),
|
||||
registry(root),
|
||||
)
|
||||
expect(response.status).toBe(416)
|
||||
expect(response.headers.get('content-range')).toBe(`bytes */${PNG_BYTES.length}`)
|
||||
})
|
||||
|
||||
it('answers HEAD without a body', async () => {
|
||||
const path = join(root, 'graph.png')
|
||||
await writeFile(path, PNG_BYTES)
|
||||
const response = await serveMediaReference(
|
||||
apiRequest(path, { method: 'HEAD' }),
|
||||
registry(root),
|
||||
)
|
||||
expect(response.status).toBe(200)
|
||||
expect(response.headers.get('content-length')).toBe(String(PNG_BYTES.length))
|
||||
expect(response.body).toBeNull()
|
||||
})
|
||||
|
||||
it('denies malformed, missing, and uncontained requests', async () => {
|
||||
const missing = await serveMediaReference(
|
||||
new Request('http://127.0.0.1:3080/api/file'),
|
||||
registry(root),
|
||||
)
|
||||
expect(missing.status).toBe(400)
|
||||
const relative = await serveMediaReference(
|
||||
new Request(`http://127.0.0.1:3080/api/file?path=${encodeURIComponent('x.png')}`),
|
||||
registry(root),
|
||||
)
|
||||
expect(relative.status).toBe(400)
|
||||
const gone = await serveMediaReference(
|
||||
apiRequest(join(root, 'missing.png')),
|
||||
registry(root),
|
||||
)
|
||||
expect(gone.status).toBe(404)
|
||||
expect((await serveMediaReference(apiRequest(join(root, 'x.png')), undefined)).status).toBe(403)
|
||||
|
||||
const outside = await realpath(await mkdtemp(join(tmpdir(), 'dsh-workspace-file-out-')))
|
||||
const path = join(outside, 'x.png')
|
||||
await writeFile(path, PNG_BYTES)
|
||||
try {
|
||||
const denied = await serveMediaReference(apiRequest(path), registry(root))
|
||||
expect(denied.status).toBe(403)
|
||||
expect(await denied.text()).toBe('outside workspace roots')
|
||||
} finally {
|
||||
await rm(outside, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('refuses directories and non-allowlisted or mislabeled content', async () => {
|
||||
const directory = await serveMediaReference(apiRequest(root), registry(root))
|
||||
expect(directory.status).toBe(403)
|
||||
|
||||
const text = join(root, 'note.txt')
|
||||
await writeFile(text, TEXT_BYTES)
|
||||
expect((await serveMediaReference(apiRequest(text), registry(root))).status).toBe(415)
|
||||
|
||||
const fake = join(root, 'fake.png')
|
||||
await writeFile(fake, TEXT_BYTES)
|
||||
expect((await serveMediaReference(apiRequest(fake), registry(root))).status).toBe(415)
|
||||
})
|
||||
|
||||
it('follows a symlink into the workspace for the containment check', async () => {
|
||||
const real = join(root, 'real.png')
|
||||
await writeFile(real, PNG_BYTES)
|
||||
const link = join(root, 'link.png')
|
||||
await symlink(real, link)
|
||||
const response = await serveMediaReference(apiRequest(link), registry(root))
|
||||
expect(response.status).toBe(200)
|
||||
expect(await responseBytes(response)).toEqual(PNG_BYTES)
|
||||
})
|
||||
|
||||
it('survives a large sparse file check without buffering it whole', async () => {
|
||||
const path = join(root, 'huge.mp4')
|
||||
const handle = await open(path, 'w')
|
||||
try {
|
||||
await handle.truncate(256 * 1024 * 1024)
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
const response = await serveMediaReference(
|
||||
apiRequest(path, { headers: { range: 'bytes=0-9' } }),
|
||||
registry(root),
|
||||
)
|
||||
expect(response.status).toBe(206)
|
||||
expect(response.headers.get('content-length')).toBe('10')
|
||||
expect((await responseBytes(response)).length).toBe(10)
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionMediaReferences plugin contribution', () => {
|
||||
const roots: Context[] = []
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(roots.splice(0).map(ctx => ctx.fiber.dispose()))
|
||||
})
|
||||
|
||||
it('registers the /api/file route when connection and registry are composed', async () => {
|
||||
const unregister = vi.fn(() => {})
|
||||
let registered: {
|
||||
path: string
|
||||
methods: readonly string[]
|
||||
requestBody: 'buffered'
|
||||
fetch: (request: Request) => Promise<Response>
|
||||
} | undefined
|
||||
const register = vi.fn((route: typeof registered) => {
|
||||
registered = route
|
||||
return unregister
|
||||
})
|
||||
const ctx = new Context()
|
||||
roots.push(ctx)
|
||||
ctx.provide('connection', { fetch: { register } } as never)
|
||||
ctx.provide('workspaceRegistry', { list: () => [] } as never)
|
||||
await ctx.plugin(SessionMediaReferences).await()
|
||||
|
||||
expect(register).toHaveBeenCalledTimes(1)
|
||||
expect(registered?.path).toBe('/api/file')
|
||||
expect(registered?.methods).toEqual(['GET', 'HEAD'])
|
||||
expect(registered?.requestBody).toBe('buffered')
|
||||
|
||||
// The composed route consults the composed registry: an empty registry
|
||||
// refuses an existing absolute path with the containment verdict.
|
||||
const existing = await realpath(tmpdir())
|
||||
const denied = await registered?.fetch(
|
||||
new Request(`http://127.0.0.1:3080/api/file?path=${encodeURIComponent(existing)}`),
|
||||
)
|
||||
expect(denied?.status).toBe(403)
|
||||
expect(await denied?.text()).toBe('outside workspace roots')
|
||||
|
||||
await ctx.fiber.dispose()
|
||||
expect(unregister).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
})
|
||||
@@ -17,6 +17,7 @@
|
||||
"src/file-references.ts",
|
||||
"src/history.ts",
|
||||
"src/list.ts",
|
||||
"src/media-references.ts",
|
||||
"src/model-selection-projection.ts",
|
||||
"src/skill-catalog.ts"
|
||||
],
|
||||
@@ -30,6 +31,7 @@
|
||||
{ "path": "../../context/file-reference" },
|
||||
{ "path": "../../attachment/attachment" },
|
||||
{ "path": "../../client/file-upload/tsconfig.host.json" },
|
||||
{ "path": "../../client/connection/tsconfig.host.json" },
|
||||
{ "path": "../../interaction/permission-presets" },
|
||||
{ "path": "../../jobs/jobs" },
|
||||
{ "path": "../../llm/llm" },
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
import { Fragment, memo, useMemo } from 'react'
|
||||
import type { ReactNode } from 'react'
|
||||
import { JsonBlock, MarkdownText } from '@deepseek-ai/dsh-client-ui-primitives'
|
||||
import type { MarkdownFileMentions } from '@deepseek-ai/dsh-client-ui-primitives'
|
||||
import type { MarkdownFileMentions, MarkdownPathImages } from '@deepseek-ai/dsh-client-ui-primitives'
|
||||
import type { ChatNodeOwnerProps, ChatViewSlotProps } from '../contract/slots.ts'
|
||||
import type { AssistantBlock } from '../contract/snapshot.ts'
|
||||
import { markdownLabels } from '../markdown-labels.ts'
|
||||
import { ReasoningRow } from './ReasoningRow.tsx'
|
||||
import { localPathMediaUrl } from './local-path-media.ts'
|
||||
import { useSearchableHidden } from './searchable-hidden.ts'
|
||||
import css from './AssistantMarkdown.module.css'
|
||||
|
||||
@@ -34,6 +35,13 @@ export const AssistantMarkdown = memo(function AssistantMarkdown({
|
||||
// Stable per locale revision (t identity changes on switch): a fresh object
|
||||
// per render would rebuild MarkdownText's component table every chunk.
|
||||
const labels = useMemo(() => markdownLabels(t), [t])
|
||||
// Local media paths in the closing prose rewrite to the same-origin file
|
||||
// API (policy re-validation lives host-side). The vocabulary identity is
|
||||
// stable per page load because MarkdownText memoizes on it.
|
||||
const pathImages = useMemo<MarkdownPathImages>(() => {
|
||||
const { protocol, origin } = window.location
|
||||
return { resolve: value => localPathMediaUrl(protocol, origin, value) }
|
||||
}, [])
|
||||
const last = blocks.length - 1
|
||||
// Tool-call heads render as tool rows in the chat view's grouping pass, so
|
||||
// a node that is only those heads (or empty) would paint an empty root
|
||||
@@ -55,6 +63,7 @@ export const AssistantMarkdown = memo(function AssistantMarkdown({
|
||||
streaming={streaming}
|
||||
labels={labels}
|
||||
fileMentions={mentions}
|
||||
pathImages={pathImages}
|
||||
/>,
|
||||
)
|
||||
break
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
/**
|
||||
* Pure local-path → same-origin workspace-file mapping for closing prose. The
|
||||
* renderer seam consumes this vocabulary; the served endpoint re-validates
|
||||
* every request, so this side only decides whether a destination *looks*
|
||||
* like an absolute local path the Host could serve. One endpoint covers every
|
||||
* served media type (images today; video/audio the same way), so consumers
|
||||
* rewrite the path without knowing what kind of media it names.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Map one authored media destination to the same-origin workspace-file URL.
|
||||
* @param protocol - `window.location.protocol` at render time.
|
||||
* @param origin - `window.location.origin` at render time.
|
||||
* @param value - The authored markdown destination, exactly as written.
|
||||
* @returns The API URL for an absolute POSIX path on an HTTP(S) page, or
|
||||
* undefined when the destination cannot be a Host-served local file
|
||||
* (non-HTTP transport such as Electron `file://`, protocol-relative or
|
||||
* relative destinations).
|
||||
*/
|
||||
export function localPathMediaUrl(protocol: string, origin: string, value: string): string | undefined {
|
||||
if (protocol !== 'http:' && protocol !== 'https:') return undefined
|
||||
if (value.length === 0 || !value.startsWith('/') || value.startsWith('//')) return undefined
|
||||
return `${origin}/api/file?path=${encodeURIComponent(value)}`
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
// @vitest-environment jsdom
|
||||
import { cleanup, render } from '@testing-library/react'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { AssistantMarkdown } from '../src/client/chat/AssistantMarkdown.tsx'
|
||||
import type { ChatNodeOwnerProps, ChatViewSlotProps } from '../src/client/contract/slots.ts'
|
||||
import type { AssistantBlock } from '../src/client/contract/snapshot.ts'
|
||||
|
||||
afterEach(cleanup)
|
||||
|
||||
const t = ((_key: string) => 'label') as unknown as ChatViewSlotProps['t']
|
||||
const renderMessageImages = (() => null) as unknown as ChatNodeOwnerProps['renderMessageImages']
|
||||
|
||||
function textBlock(text: string): AssistantBlock {
|
||||
return { kind: 'text', text }
|
||||
}
|
||||
|
||||
describe('AssistantMarkdown local-path images', () => {
|
||||
it('renders a local image path in closing prose through the same-origin API', () => {
|
||||
const { container } = render(
|
||||
<AssistantMarkdown
|
||||
blocks={[textBlock('See  for the layout.')]}
|
||||
streaming={false}
|
||||
renderMessageImages={renderMessageImages}
|
||||
t={t}
|
||||
/>,
|
||||
)
|
||||
const image = container.querySelector('img')
|
||||
expect(image?.getAttribute('alt')).toBe('diagram')
|
||||
const url = new URL(image?.getAttribute('src') ?? '')
|
||||
expect(url.pathname).toBe('/api/file')
|
||||
expect(url.searchParams.get('path')).toBe('/tmp/graph.png')
|
||||
})
|
||||
|
||||
it('keeps non-absolute destinations inert', () => {
|
||||
const { container } = render(
|
||||
<AssistantMarkdown
|
||||
blocks={[textBlock('See .')]}
|
||||
streaming={false}
|
||||
renderMessageImages={renderMessageImages}
|
||||
t={t}
|
||||
/>,
|
||||
)
|
||||
expect(container.querySelector('img')).toBeNull()
|
||||
expect(container.textContent).toContain('diagram')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,30 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { localPathMediaUrl } from '../src/client/chat/local-path-media.ts'
|
||||
|
||||
const ORIGIN = 'http://127.0.0.1:3080'
|
||||
|
||||
describe('localPathMediaUrl', () => {
|
||||
it('maps an absolute POSIX path on an HTTP page to the image API', () => {
|
||||
expect(localPathMediaUrl('http:', ORIGIN, '/tmp/graph.png'))
|
||||
.toBe(`${ORIGIN}/api/file?path=${encodeURIComponent('/tmp/graph.png')}`)
|
||||
expect(localPathMediaUrl('https:', 'https://127.0.0.1:3080', '/tmp/graph.png'))
|
||||
.toBe(`https://127.0.0.1:3080/api/file?path=${encodeURIComponent('/tmp/graph.png')}`)
|
||||
})
|
||||
|
||||
it('keeps non-HTTP transports inert', () => {
|
||||
expect(localPathMediaUrl('file:', 'file:///app', '/tmp/graph.png')).toBeUndefined()
|
||||
expect(localPathMediaUrl('ws:', ORIGIN, '/tmp/graph.png')).toBeUndefined()
|
||||
})
|
||||
|
||||
it('keeps destinations that cannot be Host-served local files inert', () => {
|
||||
expect(localPathMediaUrl('http:', ORIGIN, '')).toBeUndefined()
|
||||
expect(localPathMediaUrl('http:', ORIGIN, '//cdn.example.com/x.png')).toBeUndefined()
|
||||
expect(localPathMediaUrl('http:', ORIGIN, 'relative.png')).toBeUndefined()
|
||||
expect(localPathMediaUrl('http:', ORIGIN, 'C:\\tmp\\x.png')).toBeUndefined()
|
||||
})
|
||||
|
||||
it('encodes the full path including spaces', () => {
|
||||
expect(localPathMediaUrl('http:', ORIGIN, '/tmp/my graph.png'))
|
||||
.toBe(`${ORIGIN}/api/file?path=${encodeURIComponent('/tmp/my graph.png')}`)
|
||||
})
|
||||
})
|
||||
@@ -64,7 +64,7 @@ export { CodeBlock } from './markdown/CodeBlock.tsx'
|
||||
export type { CodeBlockProps } from './markdown/CodeBlock.tsx'
|
||||
export { JsonBlock } from './markdown/JsonBlock.tsx'
|
||||
export { MarkdownText } from './markdown/MarkdownText.tsx'
|
||||
export type { MarkdownCodeLabels, MarkdownFileMentions, MarkdownLabels } from './markdown/MarkdownText.tsx'
|
||||
export type { MarkdownCodeLabels, MarkdownFileMentions, MarkdownLabels, MarkdownPathImages } from './markdown/MarkdownText.tsx'
|
||||
export { extractMarkdownPlainText } from './markdown/plain-text.ts'
|
||||
export type { MarkdownPlainTextMode, MarkdownPlainTextOptions } from './markdown/plain-text.ts'
|
||||
export * from './icons/index.tsx'
|
||||
@@ -19,17 +19,18 @@ import {
|
||||
collectReferenceTargets, createReferenceTargets, renderBlocks, renderFootnoteSection,
|
||||
wrapBlockChildren,
|
||||
} from './render.tsx'
|
||||
import type { MarkdownFileMentions, MarkdownLabels, MarkdownRenderContext, ReferenceTargets } from './render.tsx'
|
||||
import type { MarkdownFileMentions, MarkdownLabels, MarkdownPathImages, MarkdownRenderContext, ReferenceTargets } from './render.tsx'
|
||||
import 'katex/dist/katex.min.css'
|
||||
import css from './MarkdownText.module.css'
|
||||
|
||||
export type { MarkdownCodeLabels, MarkdownFileMentions, MarkdownLabels } from './render.tsx'
|
||||
export type { MarkdownCodeLabels, MarkdownFileMentions, MarkdownLabels, MarkdownPathImages } from './render.tsx'
|
||||
|
||||
/** One settled full render: parse with math, resolve references, append the footnote section. */
|
||||
function renderSettled(
|
||||
text: string,
|
||||
labels: MarkdownLabels,
|
||||
fileMentions: MarkdownFileMentions | undefined,
|
||||
pathImages: MarkdownPathImages | undefined,
|
||||
): ReactNode[] {
|
||||
const root = parseGfmWithMath(text)
|
||||
const targets = createReferenceTargets()
|
||||
@@ -38,6 +39,7 @@ function renderSettled(
|
||||
streaming: false,
|
||||
labels,
|
||||
fileMentions,
|
||||
pathImages,
|
||||
targets,
|
||||
footnoteOrder: [],
|
||||
footnoteCounts: new Map(),
|
||||
@@ -106,6 +108,7 @@ class StreamingRenderer {
|
||||
streaming: true,
|
||||
labels: this.labels,
|
||||
fileMentions: undefined,
|
||||
pathImages: undefined,
|
||||
targets: frameTargets,
|
||||
footnoteOrder: this.frozenFootnoteOrder,
|
||||
footnoteCounts: this.frozenFootnoteCounts,
|
||||
@@ -124,6 +127,7 @@ class StreamingRenderer {
|
||||
streaming: true,
|
||||
labels: this.labels,
|
||||
fileMentions: undefined,
|
||||
pathImages: undefined,
|
||||
targets: frameTargets,
|
||||
footnoteOrder: [...this.frozenFootnoteOrder],
|
||||
footnoteCounts: new Map(this.frozenFootnoteCounts),
|
||||
@@ -150,32 +154,35 @@ class StreamingRenderer {
|
||||
* `labels` forwards localized fence and footnote chrome — pass a
|
||||
* reference-stable object (memoized per locale revision), because a new
|
||||
* identity discards the streaming render cache mid-message. `fileMentions`
|
||||
* links inline-code tokens its resolver recognizes as real files; this is
|
||||
* the single streaming gate — it applies to settled renders only, because a
|
||||
* links inline-code tokens its resolver recognizes as real files, and
|
||||
* `pathImages` rewrites image destinations that are local file paths into
|
||||
* displayable URLs its resolver vouches for; both vocabularies are the
|
||||
* single streaming gate — they apply to settled renders only, because a
|
||||
* streaming message's vocabulary is not final and frozen cached elements
|
||||
* must not bake in handlers that could go stale.
|
||||
* @returns A GFM document with TeX math rendered through KaTeX; raw HTML,
|
||||
* relative links, and unsafe protocols are disabled, while absolute HTTP(S)
|
||||
* images render directly.
|
||||
*/
|
||||
export const MarkdownText = memo(function MarkdownText({ text, streaming = false, labels, fileMentions }: {
|
||||
export const MarkdownText = memo(function MarkdownText({ text, streaming = false, labels, fileMentions, pathImages }: {
|
||||
text: string
|
||||
streaming?: boolean
|
||||
labels: MarkdownLabels
|
||||
fileMentions?: MarkdownFileMentions | undefined
|
||||
pathImages?: MarkdownPathImages | undefined
|
||||
}) {
|
||||
const streamRef = useRef<StreamingRenderer | null>(null)
|
||||
const streamLabelsRef = useRef<MarkdownLabels>(labels)
|
||||
const children = useMemo(() => {
|
||||
if (!streaming) {
|
||||
streamRef.current = null
|
||||
return renderSettled(text, labels, fileMentions)
|
||||
return renderSettled(text, labels, fileMentions, pathImages)
|
||||
}
|
||||
if (streamRef.current === null || streamLabelsRef.current !== labels) {
|
||||
streamRef.current = new StreamingRenderer(labels)
|
||||
streamLabelsRef.current = labels
|
||||
}
|
||||
return streamRef.current.render(text)
|
||||
}, [text, streaming, labels, fileMentions])
|
||||
}, [text, streaming, labels, fileMentions, pathImages])
|
||||
return <div className={css.markdown}>{children}</div>
|
||||
})
|
||||
@@ -69,6 +69,35 @@ function remoteImageUrl(url: string): string | undefined {
|
||||
}
|
||||
}
|
||||
|
||||
/** Protocols a vocabulary-rewritten image destination may carry. */
|
||||
function vocabularyImageUrl(url: string): string | undefined {
|
||||
try {
|
||||
const protocol = new URL(url).protocol
|
||||
return protocol === 'http:' || protocol === 'https:' || protocol === 'blob:' || protocol === 'data:'
|
||||
? url
|
||||
: undefined
|
||||
} catch {
|
||||
// A vocabulary result must be an absolute URL; anything else stays a miss.
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The displayable source for one image destination: absolute HTTP(S) as
|
||||
* authored, otherwise the context's local-path vocabulary when it vouches for
|
||||
* the destination. Either miss leaves the authored fallback (alt text) to the
|
||||
* caller.
|
||||
* @param url - The authored markdown destination.
|
||||
* @param pathImages - Rewriting vocabulary, when the render pass has one.
|
||||
* @returns The displayable image URL, or undefined.
|
||||
*/
|
||||
function imageSource(url: string, pathImages: MarkdownPathImages | undefined): string | undefined {
|
||||
const remote = remoteImageUrl(sanitizeUrl(normalizeUri(url)))
|
||||
if (remote !== undefined) return remote
|
||||
const rewritten = pathImages?.resolve(url)
|
||||
return rewritten === undefined ? undefined : vocabularyImageUrl(rewritten)
|
||||
}
|
||||
|
||||
/** Link/image reference targets collected from a document (first definition per identifier wins, as in CommonMark). */
|
||||
export interface ReferenceTargets {
|
||||
/** Link/image definitions keyed by upper-cased identifier. */
|
||||
@@ -107,6 +136,24 @@ export function collectReferenceTargets(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Local-path image vocabulary for image destinations: the owner maps an
|
||||
* authored destination that fails the remote-URL allowlist (an absolute local
|
||||
* file path, for example) to a displayable URL it can vouch for. Absent
|
||||
* wherever no such vocabulary exists, authored local destinations keep their
|
||||
* documented fallback (the image's alt text). Rewritten destinations must be
|
||||
* absolute; the renderer re-checks their protocol before emitting them.
|
||||
*/
|
||||
export interface MarkdownPathImages {
|
||||
/**
|
||||
* Resolve one authored image destination.
|
||||
* @param value - The destination exactly as the markdown author wrote it.
|
||||
* @returns A displayable absolute URL, or undefined when the destination
|
||||
* names no displayable image — it then stays inert alt text.
|
||||
*/
|
||||
resolve(value: string): string | undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* File-mention affordance for inline code: the owner resolves an authored
|
||||
* token to the file it names, using its own vocabulary of real files — the
|
||||
@@ -135,6 +182,8 @@ export interface MarkdownRenderContext {
|
||||
readonly inBlockquote?: boolean
|
||||
/** Inline-code file mentions; absent wherever no opener vocabulary exists. */
|
||||
readonly fileMentions: MarkdownFileMentions | undefined
|
||||
/** Local-path image vocabulary; absent wherever no rewriting owner exists. */
|
||||
readonly pathImages: MarkdownPathImages | undefined
|
||||
/** Inside an anchor's children: interactive mentions must not nest there. */
|
||||
readonly inLink?: boolean
|
||||
/** Reference targets visible to this pass. */
|
||||
@@ -293,7 +342,7 @@ function renderNode(node: Md.RootContent, key: Key, context: MarkdownRenderConte
|
||||
case 'linkReference':
|
||||
return renderLinkReference(node, key, context)
|
||||
case 'image':
|
||||
return renderImage(node.url, node.alt ?? '', key)
|
||||
return renderImage(node.url, node.alt ?? '', key, context)
|
||||
case 'imageReference':
|
||||
return renderImageReference(node, key, context)
|
||||
case 'footnoteReference':
|
||||
@@ -513,8 +562,8 @@ function inlineCodeHttpUrl(value: string): string | undefined {
|
||||
}
|
||||
}
|
||||
|
||||
function renderImage(url: string, alt: string, key: Key): ReactNode {
|
||||
const imageSrc = remoteImageUrl(sanitizeUrl(normalizeUri(url)))
|
||||
function renderImage(url: string, alt: string, key: Key, context: MarkdownRenderContext): ReactNode {
|
||||
const imageSrc = imageSource(url, context.pathImages)
|
||||
if (imageSrc === undefined) {
|
||||
return <span key={key} className={css.imageAlt}>{alt}</span>
|
||||
}
|
||||
@@ -562,7 +611,7 @@ function renderImageReference(
|
||||
): ReactNode {
|
||||
const definition = context.targets.definitions.get(node.identifier.toUpperCase())
|
||||
if (definition === undefined) return `'
|
||||
|
||||
const mapping = (value: string): string | undefined =>
|
||||
value === '/tmp/graph.png' ? 'https://cdn.example.com/graph.png' : undefined
|
||||
|
||||
describe('MarkdownText local-path images', () => {
|
||||
it('renders authored alt text when no path vocabulary exists', () => {
|
||||
const { container } = render(<MarkdownText text={LOCAL_IMAGE} />)
|
||||
expect(container.querySelector('img')).toBeNull()
|
||||
expect(screen.getByText('diagram')).toBeTruthy()
|
||||
})
|
||||
|
||||
it('rewrites a local image path through the vocabulary', () => {
|
||||
const pathImages: MarkdownPathImages = { resolve: mapping }
|
||||
const { container } = render(<MarkdownText text={LOCAL_IMAGE} pathImages={pathImages} />)
|
||||
const image = container.querySelector('img')
|
||||
expect(image?.getAttribute('src')).toBe('https://cdn.example.com/graph.png')
|
||||
expect(image?.getAttribute('alt')).toBe('diagram')
|
||||
})
|
||||
|
||||
it('keeps the alt fallback when the vocabulary misses', () => {
|
||||
const pathImages: MarkdownPathImages = { resolve: () => undefined }
|
||||
const { container } = render(<MarkdownText text={LOCAL_IMAGE} pathImages={pathImages} />)
|
||||
expect(container.querySelector('img')).toBeNull()
|
||||
expect(screen.getByText('diagram')).toBeTruthy()
|
||||
})
|
||||
|
||||
it('accepts data and blob vocabulary results', () => {
|
||||
const data: MarkdownPathImages = { resolve: () => 'data:image/png;base64,AAAA' }
|
||||
const blob: MarkdownPathImages = { resolve: () => 'blob:https://example.com/id' }
|
||||
const dataRender = render(<MarkdownText text={LOCAL_IMAGE} pathImages={data} />)
|
||||
const blobRender = render(<MarkdownText text={LOCAL_IMAGE} pathImages={blob} />)
|
||||
expect(dataRender.container.querySelector('img')?.getAttribute('src'))
|
||||
.toBe('data:image/png;base64,AAAA')
|
||||
expect(blobRender.container.querySelector('img')?.getAttribute('src'))
|
||||
.toBe('blob:https://example.com/id')
|
||||
})
|
||||
|
||||
it('rejects non-absolute vocabulary results', () => {
|
||||
for (const result of ['relative.png', '/api/image?path=%2Ftmp%2Fx.png', 'ftp://host/x.png']) {
|
||||
const pathImages: MarkdownPathImages = { resolve: () => result }
|
||||
const { container, unmount } = render(<MarkdownText text={LOCAL_IMAGE} pathImages={pathImages} />)
|
||||
expect(container.querySelector('img')).toBeNull()
|
||||
unmount()
|
||||
}
|
||||
})
|
||||
|
||||
it('leaves remote images untouched even when the vocabulary maps them', () => {
|
||||
const remote = ''
|
||||
const pathImages: MarkdownPathImages = { resolve: () => 'https://cdn.example.com/b.png' }
|
||||
const { container } = render(<MarkdownText text={remote} pathImages={pathImages} />)
|
||||
expect(container.querySelector('img')?.getAttribute('src')).toBe('https://example.com/a.png')
|
||||
})
|
||||
|
||||
it('rewrites reference-style local image destinations', () => {
|
||||
const reference = ['![diagram][fig]', '', '[fig]: /tmp/graph.png'].join('\n')
|
||||
const pathImages: MarkdownPathImages = { resolve: mapping }
|
||||
const { container } = render(<MarkdownText text={reference} pathImages={pathImages} />)
|
||||
expect(container.querySelector('img')?.getAttribute('src'))
|
||||
.toBe('https://cdn.example.com/graph.png')
|
||||
})
|
||||
|
||||
it('applies the vocabulary only to settled renders, never while streaming', () => {
|
||||
const pathImages: MarkdownPathImages = { resolve: mapping }
|
||||
const { container, rerender } = render(
|
||||
<MarkdownText text={LOCAL_IMAGE} streaming pathImages={pathImages} />,
|
||||
)
|
||||
// Streaming messages may still grow, so their prose keeps the inert alt
|
||||
// fallback until the settled pass self-heals it.
|
||||
expect(container.querySelector('img')).toBeNull()
|
||||
rerender(<MarkdownText text={LOCAL_IMAGE} pathImages={pathImages} />)
|
||||
expect(container.querySelector('img')?.getAttribute('src'))
|
||||
.toBe('https://cdn.example.com/graph.png')
|
||||
})
|
||||
})
|
||||
@@ -22,6 +22,7 @@ function makeContext(): MarkdownRenderContext {
|
||||
streaming: false,
|
||||
labels: markdownLabels,
|
||||
fileMentions: undefined,
|
||||
pathImages: undefined,
|
||||
targets: createReferenceTargets(),
|
||||
footnoteOrder: [],
|
||||
footnoteCounts: new Map(),
|
||||
|
||||
Reference in new issue
Block a user