diff --git a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.i18n.yaml b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.i18n.yaml index ecb554c14d..6ccdf5d446 100644 --- a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md -2026-09-08-document-preview-operations.md: 0b90c17ba94d0c7cd30afb0a6e6a033f8f12d0df -2026-09-08-document-preview-operations.zh.md: ec9c208c97dfe75afdb0915c655d9c1c7b4d42ff +2026-09-08-document-preview-operations.md: 220db6db0474f244f902cdeb89b297856243672b +2026-09-08-document-preview-operations.zh.md: 7b838711e5118b8fcc072e4efb753abfc9756b56 diff --git a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md index 0b90c17ba9..220db6db04 100644 --- a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md +++ b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md @@ -24,7 +24,7 @@ PDF.js's official TextLayerBuilder owns selection boundaries and copy normalizat ## Alternatives considered -**Load converted content through callbacks in preview metadata.** A callback makes the shared file store hold both original file bytes and format-specific conversion results. Renderer-owned loading keeps conversion caches, failures, and font metadata with Office while preserving shared file identity and toolbar controls. Definitions declare `loading: 'renderer'`; the body receives a format-independent loading revision and reports only its displayed source version. Reload or implementation replacement advances the revision, stale reports are ignored, and the body cancels pending work on replacement or unmount. Office retains settled contents for the tab lifetime and composes its own PDF child slot. +**Load converted content through callbacks in preview metadata.** A callback makes the shared file store hold both original file bytes and format-specific conversion results. Renderer-owned loading keeps conversion caches, failures, and font metadata with Office while preserving shared file identity and toolbar controls. Definitions declare `loading: 'renderer'`; the body receives a format-independent loading revision and delegates reads and store updates to its private injected face. The face reports the displayed source version or a failed load to the shared owner. Reload or implementation replacement advances the revision, stale reports are ignored, and the body cancels pending work on replacement or unmount. Office retains settled contents for the tab lifetime and composes its own PDF child slot. **Methods attached to an Iterator or its values.** This conflates observation with commands and repeats capability identity in data frames. Frames carry data and failures; explicit Preview RPC callbacks perform reads. diff --git a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.zh.md b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.zh.md index ec9c208c97..7b838711e5 100644 --- a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.zh.md +++ b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.zh.md @@ -24,7 +24,7 @@ PDF.js 官方 TextLayerBuilder 在适配宽度的 canvas 上负责选择边界 ## 考虑过的替代方案 -**通过预览元数据中的回调加载转换内容。** 这种回调会让共享文件 store 同时持有源文件字节和格式专属的转换结果。由渲染器自行加载可将转换缓存、错误和字体元数据留在 Office，同时保留共享文件身份和工具栏控件。定义声明 `loading: 'renderer'`；正文接收与格式无关的加载 revision，只报告已展示的源版本。重新加载或替换实现会增加 revision，过期报告会被忽略，正文在替换或卸载时取消待处理工作。Office 在 tab 生命周期内保留已完成的内容，并组合自己的 PDF 子 slot。 +**通过预览元数据中的回调加载转换内容。** 这种回调会让共享文件 store 同时持有源文件字节和格式专属的转换结果。由渲染器自行加载可将转换缓存、错误和字体元数据留在 Office，同时保留共享文件身份和工具栏控件。定义声明 `loading: 'renderer'`；正文接收与格式无关的加载 revision，并将读取与 store 更新交给其私有注入 face。face 向共享 owner 报告已展示的源版本或加载失败。重新加载或替换实现会增加 revision，过期报告会被忽略，正文在替换或卸载时取消待处理工作。Office 在 tab 生命周期内保留已完成的内容，并组合自己的 PDF 子 slot。 **把方法挂到 Iterator 或其值上。** 这会混淆观察与命令，并在数据帧中重复能力身份。帧携带数据和失败；显式 Preview RPC 回调负责读取。 diff --git a/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.i18n.yaml b/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.i18n.yaml index b40fa370ec..8e3e128348 100644 --- a/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-11-node-office-kit.md -2026-09-11-node-office-kit.md: 132ac8b3dd709f9f16254a5569dec59f01d29fda -2026-09-11-node-office-kit.zh.md: c7bdab2322766c42cfee4092f158d7b6b530e875 +2026-09-11-node-office-kit.md: 36897178014daa839399adb5fdf72a4ca53cec8b +2026-09-11-node-office-kit.zh.md: 1255012bf6281abe9a95ac338480cf2424fcbe9a diff --git a/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.md b/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.md index 132ac8b3dd..3689717801 100644 --- a/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.md +++ b/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.md @@ -10,11 +10,11 @@ Binary Office and OOXML files require document layout before they can be preview ## Decision -The [office-to-pdf capability](../../../../packages/document/README.md) delegates conversion to the independently released `@deepseek-ai/libreoffice-kit` Node API. The [kit ownership decision](2026-09-14-independent-libreoffice-kit.md) owns source maintenance, compatibility versions, and npm distribution. DSH owns Session file authorization, conversion concurrency, private scratch files, output limits, and Remote transport. The [Web bundle](../../../../packages/bundle/web-app/README.md) declares separate provider and controller entries and the shared Document Preview entry with stable IDs. Conversion and authorized transport remain independently configurable; Office UI shares the Document Preview Loader lifetime. +The [office-to-pdf capability](../../../../packages/document/README.md) delegates conversion to the independently released `@deepseek-ai/libreoffice-kit` Node API. The [kit ownership decision](2026-09-14-independent-libreoffice-kit.md) owns source maintenance, compatibility versions, and npm distribution. DSH owns Session file authorization, conversion concurrency, private scratch files, output limits, and Remote transport. The [Web bundle](../../../../packages/bundle/web-app/README.md) declares the conversion service and shared Document Preview entry with stable IDs. The service owns authorized conversion Remote methods, whose descriptor is mounted by `api/remotes`; Office UI shares the Document Preview Loader lifetime. The [platform engine decision](2026-09-15-platform-office-engines.md) requires the kit’s declared native target engine, or WASM when no native target is declared. Missing or invalid required engines reject conversion. The shared [bounded provider](2026-09-15-bounded-office-conversion.md) owns admission, conversion reuse, and cancellation through scratch cleanup. Preview consumes that provider without registering another converter or requiring Office authoring skills. -The service and Remote methods accept DOC, DOCX, XLS, XLSX, PPT, and PPTX. The Client preview registers DOC, DOCX, PPT, and PPTX; XLS and XLSX use the unsupported-preview state and can open in the system default application without requesting conversion. The kit verifies bounded ZIP membership and content types for OOXML inputs and the OLE compound-file header for binary Office inputs before LibreOffice imports them. Renaming text to an Office suffix does not admit it. Binary formats return no missing-font diagnostics because the kit does not extract their font tables. It writes a fresh, exclusively created PDF in a caller-owned private directory. DSH reads and validates the complete output before removing scratch files. The [service’s Remote method](../../../../packages/document/office-to-pdf/README.md) authorizes source access through [Workspace Files](2026-09-09-workspace-file-read-authority.md), preserves the source path/version, and returns PDF bytes. Source read caps and generated PDF caps remain independent. One source path/version snapshot governs the access probe and deferred read, including size-failure rechecks, so conversion cannot publish bytes under another identity. Preview bytes do not enter Session storage or persistent caches. +The service and Remote methods accept DOC, DOCX, XLS, XLSX, PPT, and PPTX. Client PDF previews use this path for Word and PowerPoint; [browser spreadsheet preview](../feature/2026-09-16-browser-excel-preview.md) reads XLSX, XLS, CSV, and TSV independently. The kit verifies bounded ZIP membership and content types for OOXML inputs and the OLE compound-file header for binary Office inputs before LibreOffice imports them. Renaming text to an Office suffix does not admit it. Binary formats return no missing-font diagnostics because the kit does not extract their font tables. It writes a fresh, exclusively created PDF in a caller-owned private directory. DSH reads and validates the complete output before removing scratch files. The [service’s Remote method](../../../../packages/document/office-to-pdf/README.md) authorizes source access through [Workspace Files](2026-09-09-workspace-file-read-authority.md), preserves the source path/version, and returns PDF bytes. Source read caps and generated PDF caps remain independent. One source path/version snapshot governs the access probe and deferred read, including size-failure rechecks, so conversion cannot publish bytes under another identity. Preview bytes do not enter Session storage or persistent caches. A converter reuses font metadata obtained by its first conversion Worker; original font buffers and decoded glyph coverage remain conversion-local. Workers validate indexed files when reading them. Exact installed families precede configured alternatives, and complete family/style/weight/italic/width/pitch/language/code-point requests retain their distinct matches. WASM callbacks import original font files, including complete collections, into MEMFS. Native engines also retain their platform's font discovery. Neither path downloads or installs fonts; native OS-managed font memory is outside the explicit import budget. Recreating the converter refreshes its metadata after font changes. diff --git a/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.zh.md b/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.zh.md index c7bdab2322..1255012bf6 100644 --- a/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.zh.md +++ b/.agents/notes/implemented/architecture/2026-09-11-node-office-kit.zh.md @@ -10,11 +10,11 @@ Status: implemented ## 决策 -[文档渲染能力](../../../../packages/document/README.zh.md)将转换委托给独立发布的 `@deepseek-ai/libreoffice-kit` Node API。[kit 归属决策](2026-09-14-independent-libreoffice-kit.zh.md)负责源码维护、兼容版本和 npm 分发。DSH 负责 Session 文件授权、转换并发、私有临时文件、输出限制和 Remote 传输。[Web bundle](../../../../packages/bundle/web-app/README.zh.md)使用稳定 ID 声明独立的 provider、controller 入口和共享文档预览入口。转换与授权传输仍可独立配置；Office UI 共享文档预览的 Loader 生命周期。 +[文档渲染能力](../../../../packages/document/README.zh.md)将转换委托给独立发布的 `@deepseek-ai/libreoffice-kit` Node API。[kit 归属决策](2026-09-14-independent-libreoffice-kit.zh.md)负责源码维护、兼容版本和 npm 分发。DSH 负责 Session 文件授权、转换并发、私有临时文件、输出限制和 Remote 传输。[Web bundle](../../../../packages/bundle/web-app/README.zh.md)使用稳定 ID 声明转换服务与共享文档预览入口。服务负责带授权的转换 Remote 方法，其描述符由 `api/remotes` 挂载；Office UI 共享文档预览的 Loader 生命周期。 [平台引擎决策](2026-09-15-platform-office-engines.zh.md)要求使用 kit 已声明的原生目标引擎，未声明原生目标时使用 WASM。缺失或无效的必需引擎会拒绝转换。共享的[有界提供方](2026-09-15-bounded-office-conversion.zh.md)负责准入、转换复用以及持续到临时文件清理完成的取消。预览消费该提供方，不注册另一个转换器，也不依赖 Office 创作 skills。 -服务与 Remote 方法接受 DOC、DOCX、XLS、XLSX、PPT 和 PPTX。Client 预览注册 DOC、DOCX、PPT 和 PPTX；XLS 与 XLSX 使用不支持预览的空状态，可由系统默认应用打开，并且不请求转换。LibreOffice 导入前，kit 校验 OOXML 输入的有界 ZIP 成员和内容类型，以及二进制 Office 输入的 OLE 复合文件头。将文本改为 Office 后缀不能通过校验。kit 不提取二进制格式的字体表，因此这些格式不返回缺失字体诊断。kit 在调用方拥有的私有目录中独占创建新的 PDF。DSH 读取并校验完整输出后才删除临时文件。[服务的 Remote 方法](../../../../packages/document/office-to-pdf/README.zh.md)通过 [Workspace Files](2026-09-09-workspace-file-read-authority.zh.md)授权源文件访问，保留源路径和版本，并返回 PDF 字节。源文件读取上限与生成 PDF 上限相互独立。读取权限探测和延迟读取（包括超限失败后的复查）采用同一个源路径／版本快照，防止转换将字节发布到另一个源身份下。预览字节不会进入 Session 存储或持久缓存。 +服务和 Remote 方法接受 DOC、DOCX、XLS、XLSX、PPT 和 PPTX。Client PDF 预览通过此路径显示 Word 和 PowerPoint；[浏览器表格预览](../feature/2026-09-16-browser-excel-preview.zh.md)独立读取 XLSX、XLS、CSV 和 TSV。LibreOffice 导入前，kit 校验 OOXML 输入的有界 ZIP 成员和内容类型，以及二进制 Office 输入的 OLE 复合文件头。将文本改为 Office 后缀不能通过校验。kit 不提取二进制格式的字体表，因此这些格式不返回缺失字体诊断。kit 在调用方拥有的私有目录中独占创建新的 PDF。DSH 读取并校验完整输出后才删除临时文件。[服务的 Remote 方法](../../../../packages/document/office-to-pdf/README.zh.md)通过 [Workspace Files](2026-09-09-workspace-file-read-authority.zh.md)授权源文件访问，保留源路径和版本，并返回 PDF 字节。源文件读取上限与生成 PDF 上限相互独立。读取权限探测和延迟读取（包括超限失败后的复查）采用同一个源路径／版本快照，防止转换将字节发布到另一个源身份下。预览字节不会进入 Session 存储或持久缓存。 converter 复用首个转换 Worker 返回的字体元数据；原始字体缓冲区和解码后的字符覆盖范围仍只属于单次转换。Worker 读取字体时校验索引中的文件。已安装字体族的精确匹配优先于配置的替代字体，完整的字体族、样式、字重、斜体、宽度、字距、语言与码点请求保留各自的匹配结果。WASM 回调将包含完整字体集合的原始字体文件导入 MEMFS。原生引擎还保留各平台的字体发现能力。两条路径均不下载或安装字体；原生操作系统管理的字体内存不受显式导入预算约束。字体变化后，重新创建 converter 会刷新元数据。 diff --git a/.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.i18n.yaml b/.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.i18n.yaml new file mode 100644 index 0000000000..65566f68ce --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.md +2026-09-16-experimental-voice-input.md: 30c81ab233f27242dcf1fe2ef89bbb86542f9c04 +2026-09-16-experimental-voice-input.zh.md: 76aa8d6fc4bea32f194b20d05db4bf49af0be313 diff --git a/.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.md b/.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.md new file mode 100644 index 0000000000..30c81ab233 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.md @@ -0,0 +1,35 @@ +# Agent Note: Experimental voice input with explicit recognition providers + +Status: implemented + +English | [中文](2026-09-16-experimental-voice-input.zh.md) + +## Problem + +Dictation must preserve user control over when a coding task starts. Local recognition brings model files, a native runtime and CPU work; merely installing or enabling a UI contribution must not allocate these resources. Later cloud providers need a selection mechanism that does not silently change where recordings are processed. + +## Decision + +Five public packages under `packages/experimental/` compose one optional, default-disabled voice input bundle. Service Definition, SenseVoice Provider and Remote Consumer are separate because provider deployment and browser transport evolve independently. The browser UI dynamically mounts its generated Remote contribution. Stable API Remotes do not acquire an experimental dependency. + +Provider ids select exact registrations. Resolution captures an instance, and deregistration closes admission, cancels and joins accepted work. There is no provider fallback: a local failure cannot authorize uploading speech to a cloud service. A new cloud provider supplies its own credentials and explicit processing location through the same registry. + +Recordings and transcripts remain transient until the user submits ordinary text. Recognition does not start an agent turn or create an audio Session event. The input facade accepts asynchronous plain text against a captured draft revision, preserving reference chips and undo history. Changed drafts retain the recognized text for explicit insertion instead of replacing later edits. Session changes and plugin disposal invalidate late results and release recording resources. Capture failures notify the current recording activity immediately, independently of resource-close latency. Cancellation, failure and plugin withdrawal join one release promise, so ownership lasts until the AudioContext closes. + +SenseVoice ONNX and Silero VAD run in one managed CPU subprocess. The native sherpa-onnx package includes ONNX Runtime; explicit preparation only downloads and verifies models; subsequent calls reuse the warm model until the configured idle deadline. Every activation verifies cached assets and restores complete caches to standby without starting the worker. Readiness therefore follows disk contents rather than the lifetime of a Provider instance. Recordings wait in the bounded queue during wake-up. The worker checks readiness at execution, so queued work cannot implicitly download resources invalidated by an earlier cache inspection. Inspection uses the same owned task as preparation, so cancellation and disposal join it. Disabling the provider joins its work and process exit while retaining reusable disk assets. Native browser recording and resampling avoid an additional audio conversion executable. The authenticated private loopback protocol carries complete PCM recordings, allowing cancellation to terminate the worker without sharing a long-running inference with another request. + +The Host owns preparation independently of the view that started it. An ordered step list survives view remounts and records completion only as execution advances. Recognition weights, VAD and final verification have separate operations; downloads use pinned byte counts, while other work reports elapsed time. Preparation status and progress stay in bundle details through `plugins.bundle.config`, keyed by package name. The generic `plugins.bundle.activation` slot carries explicit enablement intent and detail navigation; the voice occupant waits for Host cache inspection and offers installation guidance only for unprepared local models. Enabling an already prepared provider does not prompt or download. A single composer activity Slot expands the voice controls while preserving the draft editor and submit action. The context meter sits below the composer; recording and transcription hide the meter along with ordinary toolbar controls until the activity closes. Shared primitives provide buttons, disclosure and status markers; only measured waveform drawing belongs to the voice UI. Recordings start on click and transcribe on Stop, with feedback inside the activity. Persistent preferences remain in plugin settings. + +## Alternatives considered + +Always loading the model on plugin activation would make a UI toggle incur substantial startup time and memory even without dictation. Python/FunASR/PyTorch adds interpreter provisioning, dependency installation and disk cost; completed ONNX artifacts and platform-native packages serve CPU dictation. End users never export models. Browser-native speech recognition does not provide a uniform local-processing guarantee. Streaming voice interaction would require partial-result revision, turn arbitration and bidirectional audio semantics that complete-recording dictation does not need. + +## Consequences + +First use needs download time and disk space. Local means the Host machine, including a remote Host. A cancelled active inference loses its warm worker and the next request reloads it. Optional preparation paths support existing offline environments without silently changing defaults. + +Verification covers exact-provider routing, registration teardown, bounded audio parsing, microphone release, stale-draft and Session-switch handling, worker queue/deadline recovery, and verified download publication. Real local inference uses the same managed worker as the product. Browser composition verifies optional enablement and draft insertion; ordinary Session replay verifies that only the submitted text reaches the model. + +Real Node 24 and Electron 44 children complete FP32 and INT8 inference. Electron copies VAD output to avoid external-buffer restrictions. Three runs of the 4.2-second public Chinese sample agree; this short sample is not a complete accuracy evaluation. Desktop microphone requests are restricted to audio from the primary application window, with macOS authorization handled by the operating system. Windows, Linux and signed installers still require platform acceptance. + +INT8 is the default to reduce download and storage cost; `precision: fp32` selects the reference weights. Public English and concatenated bilingual samples show word differences between precisions, so quantization is not treated as accuracy-equivalent. A roughly 101-second repeated-speech sample completes in both runtimes; its observed post-request RSS is not a peak-memory bound. diff --git a/.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.zh.md b/.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.zh.md new file mode 100644 index 0000000000..76aa8d6fc4 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.zh.md @@ -0,0 +1,35 @@ +# Agent Note：使用显式识别 Provider 的实验性语音输入 + +Status: implemented + +[English](2026-09-16-experimental-voice-input.md) | 中文 + +## 问题 + +听写必须保留用户对编码任务启动时机的控制。本地识别带来模型文件、原生运行时和 CPU 工作；仅安装或启用 UI 贡献不应分配这些资源。之后的云端 Provider 需要明确的选择机制，不能悄悄改变录音的处理位置。 + +## 决策 + +`packages/experimental/` 下的五个公开包组合成一个默认禁用的可选语音输入 Bundle。服务定义、SenseVoice Provider 和 Remote 消费者分开，因为 Provider 部署与浏览器传输独立演化。浏览器 UI 动态挂载其生成的 Remote 贡献。稳定 API Remotes 不引入实验性依赖。 + +Provider id 选择确切的注册。解析捕获实例，注销先拒绝新任务，再取消并等待已接收的任务。不存在 Provider 回退：本地失败不能授权将语音上传至云端服务。新增云端 Provider 通过同一注册表提供自己的凭据配置与显式处理位置。 + +录音与转写在用户提交普通文字前都是临时数据。识别不启动智能体轮次，也不创建音频 Session 事件。输入门面按照捕获的草稿版本接收异步纯文本，保留引用芯片与撤销历史。草稿变化时保留识别文字，等待显式插入，而非替换之后的编辑。切换 Session 和释放插件会使迟到结果失效并释放录音资源。采集失败立即通知当前录音活动，不受资源关闭耗时影响。取消、失败和插件撤销共同等待同一个释放 Promise，所有权持续到 AudioContext 关闭。 + +SenseVoice ONNX 与 Silero VAD 在一个托管 CPU 子进程中运行。原生 sherpa-onnx 包包含 ONNX Runtime；显式准备只下载并校验模型；后续调用复用预热模型，直到配置的空闲截止时间。每次启用都会校验缓存资源，完整缓存恢复为待唤醒状态，不启动工作进程。就绪状态因此取决于磁盘内容，而非 Provider 实例的存活时间。录音在唤醒期间进入有界队列等待。工作进程在执行时检查就绪状态，因此较早的缓存检查使资源失效后，排队任务也不能隐式下载资源。检查复用准备任务的生命周期，取消与释放都会等待检查结束。禁用 Provider 等待其任务和进程退出，同时保留可复用的磁盘资源。浏览器原生录音与重采样避免增加音频转换可执行文件。带认证的私有环回协议传输完整 PCM 录音，取消时可以终止工作进程，而不与另一请求共享长时间推理。 + +Host 独立于发起视图拥有准备任务。有序步骤列表在视图重新挂载后保留，仅随着执行推进记录完成。识别模型、VAD 与最终校验分别准备；下载使用固定字节数，其他工作报告等待时间。准备状态和进度通过以包名为 key 的 `plugins.bundle.config` 显示在详情页。通用 `plugins.bundle.activation` Slot 传递显式启用意图和详情导航；语音贡献等待 Host 缓存检查，只为尚未准备的本地模型提供安装引导。启用已准备的 Provider 不弹窗也不下载。单个输入框活动 Slot 展开语音控件，同时保留草稿编辑器和提交按钮。空闲时上下文用量按钮位于输入框下方；录音和转写期间，用量按钮与普通工具栏控件一起隐藏，关闭活动后恢复。按钮、折叠与状态标记复用共享原语，语音 UI 只负责实测波形绘制。点击开始录音，停止后转写，反馈留在活动栏内。持久化偏好仍在插件设置中。 + +## 考虑的替代方案 + +在插件启用时加载模型，会让一个 UI 开关在尚未听写时就产生大量启动时间和内存占用。Python/FunASR/PyTorch 增加解释器、依赖安装与磁盘成本；ONNX 成品和平台原生包满足 CPU 听写。用户端不执行模型导出。浏览器原生语音识别不能提供一致的本地处理保证。流式语音交互需要部分结果修订、轮次仲裁与双向音频语义，而完整录音听写无需这些机制。 + +## 影响与验证 + +首次使用需要下载时间和磁盘空间。本地指 Host 所在机器，包括远程 Host。取消运行中的推理会丢失预热进程，下次请求重新加载。可选准备路径支持已有离线环境，不会悄悄改变默认行为。 + +验证覆盖精确 Provider 路由、注册释放、有界音频解析、麦克风释放、过期草稿和 Session 切换处理、工作进程队列与截止时间恢复，以及经过校验的下载发布。真实本地推理使用与产品相同的托管工作进程。浏览器组合验证可选启用与草稿插入；普通 Session 回放验证只有提交的文字进入模型。 + +Node 24 与 Electron 44 的真实子进程均完成 FP32 和 INT8 识别。Electron 的 VAD 输出使用复制缓冲区，避免 external buffer 限制。4.2 秒公开中文样本的三次识别结果一致；该短样本不代表完整准确率评测。桌面麦克风请求限制为主应用窗口的音频请求，macOS 权限由系统处理。Windows、Linux 真实机器及签名后的安装包仍需平台验收。 + +INT8 以降低下载和存储成本为默认值，`precision: fp32` 选择参考权重。公开英文与拼接的中英双语样本显示两种精度存在用词差异，因此不将量化视为准确率等效。约 101 秒的重复语音样本在两种运行时均完成识别；请求后的 RSS 观测不构成峰值内存上限。 diff --git a/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.i18n.yaml b/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.i18n.yaml index a005e6d029..ecf9b0571e 100644 --- a/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.md -2026-09-18-agent-teams-single-bundle.md: 21878a8e481871f5b0eeebd8f1bea707411ff3b7 -2026-09-18-agent-teams-single-bundle.zh.md: 39c6bfb7b365a45ff98d29f91a4d7905b26db008 +2026-09-18-agent-teams-single-bundle.md: b417061d24928877d8d31739e1e3bde6b8512d51 +2026-09-18-agent-teams-single-bundle.zh.md: ed20dac79f39f6648767d57556ed97b42d0e1c65 diff --git a/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.md b/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.md index 21878a8e48..b417061d24 100644 --- a/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.md +++ b/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.md @@ -30,4 +30,4 @@ Existing profiles that select the removed Web bundle have an upgrade compatibili ## Verification -The plugin-manager browser test observes the Team controls appear after one switch, reads the roster and task board through Remote, and observes their removal after disabling the bundle. The Team-panel browser test covers task creation; the built headless CLI test covers delegation without a Web server. These scenarios do not cover upgrading a profile that still selects the removed Web bundle. +The plugin-manager browser test observes the Team controls appear after one switch, reads the roster and task board through Remote, and observes their removal after disabling the bundle. The Team-panel browser test covers the read-only task board; the built headless CLI test covers delegation without a Web server. These scenarios do not cover upgrading a profile that still selects the removed Web bundle. diff --git a/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.zh.md b/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.zh.md index 39c6bfb7b3..ed20dac79f 100644 --- a/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.zh.md +++ b/.agents/notes/implemented/architecture/2026-09-18-agent-teams-single-bundle.zh.md @@ -30,4 +30,4 @@ Status: implemented ## 验证 -插件管理器浏览器测试观察一次开关后 Team 控件出现，通过 Remote 读取成员列表与任务看板，并观察关闭组合包后控件移除。Team 面板浏览器测试覆盖任务创建；构建后的 headless CLI 测试覆盖无需 Web 服务的委派。这些场景不覆盖仍选中已移除 Web 组合包的 profile 升级。 +插件管理器浏览器测试观察一次开关后 Team 控件出现，通过 Remote 读取成员列表与任务看板，并观察关闭组合包后控件移除。Team 面板浏览器测试覆盖只读任务看板；构建后的 headless CLI 测试覆盖无需 Web 服务的委派。这些场景不覆盖仍选中已移除 Web 组合包的 profile 升级。 diff --git a/.agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.i18n.yaml b/.agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.i18n.yaml new file mode 100644 index 0000000000..d8c8fb934a --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.md +2026-09-21-conversation-build-groups.md: 6591ebea61d8edd06edbaf5f54f2bebd1bd00602 +2026-09-21-conversation-build-groups.zh.md: 70657ded8e04f0badb987ada965aef1184cfc7b0 diff --git a/.agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.md b/.agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.md new file mode 100644 index 0000000000..6591ebea61 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.md @@ -0,0 +1,155 @@ +# Agent Note: Definition-owned grouping with generic Node/Group assembly + +Status: implemented + +English | [中文](2026-09-21-conversation-build-groups.zh.md) + +## Problem + +Chat needs expandable groups for continuous process content while keeping replies, user input, and Turn controls independent. Compact, Detailed, and Expanded must share grouping results; changing modes must not rebuild React instances by removing group containers, moving member parents, or changing keys. + +A Chat Builder aggregates one target's Nodes and indexes. Hardcoding a Chat process projector there makes the Builder own business segmentation, even if some caching moves into generic infrastructure. Business Definitions are the registration point for these rules. + +Process groups are orthogonal to Steps: one Assistant Node can contribute reasoning inside a group and a reply outside it; tools after that reply enter a later group. Step-number ranges cannot describe membership. Existing event Definitions interpret one event at a time, whereas grouping needs the Nodes those Definitions have already interpreted. + +## Decision + +The grouping foundation provides a separate, Node-input `ConversationGroupDefinition`, its registry and Session-local context, generic Builder input/publication, keyed Group storage, and the two React branches. Production Chat segmentation and PR #4565 presentation are not registered by this foundation; the [business migration proposal](../../proposed/feature/2026-09-20-chat-work-details-migration.md) retains their requirements. The [subsystem reference](../../../../docs/subsystems/conversation.md#group-definitions) and [source types](../../../../packages/client/ui-conversation/src/client/contract/groups.ts) own the current API details. + +### Responsibilities + +| Requirement | Consequence | +|---|---| +| Business grouping belongs to a registered Definition | Membership, segmentation, replies, steering, retries, summaries, and cache invalidation stay together. | +| Builder and assembler do generic work | Supply inputs, dispatch Definitions, validate references, reuse identities, and publish; never construct a concrete grouping class. | +| PR #4565 defines product behavior | Preserve its behavior without copying its invasive implementation. | +| React has two root kinds | The ordered root contains NodeReference and GroupReference, not another layout object model. | +| All modes share grouping | Mode is absent from Definition input, keys, and parent selection. | +| No buildLayout | ConversationViewDefinition keeps its existing responsibilities. | +| Foundation precedes business | No production Chat grouping, summaries, notifications, or footer changes in the foundation PR. | + +### Related decisions + +| Record | Relationship | +|---|---| +| [Business Node assembly](2026-08-09-client-conversation-node-assembly.md) | Retains event matching, Contexts, Locations, one business Node per Context, and target Builders; grouping adds another input category. | +| [Presentation policy and grouping](../../proposed/architecture/2026-09-20-chat-presentation-policy-and-step-groups.md) | Retains Definition ownership, local subscriptions, and mode independence; replaces event folding and Step-range membership as the grouping mechanism. | +| [Work-details migration](../../proposed/feature/2026-09-20-chat-work-details-migration.md) | Retains product scope and migration tracking; infrastructure alone does not complete those features. | + +These records retain independent rationale and remain active. Cross-View navigation and `toolCallFocus` are outside this change; no View handles, Label Slots, or resource-navigation architecture is introduced. + +## Definition input, state, and registration + +`ConversationNodeDefinition` keeps `match/start/update/publication/buildLocationData/buildViewNode`. `ConversationViewDefinition` keeps `target/create/isActive/toolCallFocus`. Group business is registered through `ctx.uiConversation.groups.register(processGroupDefinition)`, not a callback on either existing Definition. + +- One Group Definition per target owns the complete grouping semantics. Duplicate target registration fails rather than selecting a winner or stacking strategies. +- Registration requires an existing View target, but never constructs a Builder. First activation checks the constructed Builder for `groupInput()`; missing input fails. Ungrouped targets do not require it or allocate Group state. +- Each Session owns one derived context per registered Definition. `create()` initializes its State; it is not a `turn/start` event or an event Definition's unique start Match. Definition objects do not hold mutable cross-Session state. +- `update(context, input)` returns the adopted State. `buildGroups(context)` materializes pending output without interpreting events again or advancing counts on repeated reads. Replacement input requires a complete root and group replacement so removed Nodes cannot leave dangling references; `null` on apply retains output. +- `replace` supplies target order, timeline, and synchronous `readNode`, `readTurn`, and `readPosition` readers; `apply` also supplies projected previous/current Nodes, `changedTurns`, and `changedTurnOrders`. Readers are valid during the synchronous call only and must not be retained in State. +- Node data stays in the target Node Store. Content-only updates preserve the order array. Previous values are captured before the Builder installs projected updates, including additional Nodes changed by target projections. +- `ConversationLocationIndex` accumulates changed Turns from boundaries and Location data writes. Assembler passes one drained batch to every updated target, including a Turn ending without Node upserts. Direct ungrouped Builder callers may omit this field. +- The target position index supplies `changedTurnOrders` for visible-key, owning-Turn, or immediate-neighbour changes, including both owners of a moved Node and Turns affected by adjacent unscoped Nodes. `readTurn` returns one Turn's visible keys; `readPosition` returns `GroupNodePosition` with the owning Turn and immediate previous/next keys. Turn-only reads omit interruptions by unscoped Nodes; neighbour facts let business segmentation retain them. The index provides positions, not grouping decisions. +- The generic GroupDataMap associates target and Data across registration, storage, and reading. Undeclared targets have no group payload type; business consumers do not reconstruct summary types with unknown assertions. + +## Node/Group references and updates + +`NodeKey` names an existing Context Node; `GroupKey` identifies a group within a Session, target, and Definition. `NodeReference` and `GroupReference` have equal standing in `RenderEntry`. `GroupSnapshot` stores a group's immutable data and ordered NodeReference members separately from root order. Group creation adds neither Session events nor an execution lifecycle. + +`groupPart` is a renderer-owned part such as reasoning or response; omission selects the whole Node. A part may also appear at the root. Part completeness and content ownership belong to the business renderer, not the framework. Group data describes status, counts, or summaries, without copying Node bodies, tool results, or mutable business State. A group has no renderer field, CSS, coordinates, React elements, navigation, or nested groups. + +| Update | Meaning | +|---|---| +| Omit `entries` on apply input | Keep root order; replacement input requires the complete sequence. | +| Supply `entries`, including `[]` | Replace the complete root sequence; an empty array clears it. | +| `groups: { kind: 'replace', snapshots }` | Replace complete group records, removing omitted groups. | +| `groups: { kind: 'apply', upserts, removes }` | Upsert complete snapshots and remove named GroupKeys only. | +| Data-only change | Upsert the group with new data, reuse members, omit entries. | +| Member-only change | Upsert that group with its complete new member array; no all-group replacement. | +| Root-only change | Supply entries with empty apply upserts/removes. | +| Dissolve a group | Replace root entries with its members and remove the GroupKey atomically. Source Nodes remain. | + +`entries` and `groups` are one installation batch, not independent publication channels. Removing a member reference does not delete a Node. Upserts need snapshot contents; removals need only identity. React may remount a member after a real membership change, but mode switches must not dissolve groups. + +The store validates the submitted final result before mutation: every root Group has exactly one record and root position; Node references exist; duplicate upserts/removals and simultaneous upsert/removal fail. Across all roots and members, each `(NodeKey, groupPart)` appears at most once. A whole Node cannot coexist with any of its parts, but different parts may occupy different positions. Atomic moves between positions are allowed. + +GroupStore indexes records and sources by GroupKey, not repeated array searches. Equivalent root/member arrays and unchanged group snapshots retain identity. Data-only upserts inspect only submitted snapshots, without scanning roots, unchanged members, or Nodes. Full replacement revalidates references and still aligns results by key. Definition caches own business calculation; GroupStore owns installed results and notifications. + +## Assembly order and lifecycle + +1. BoundConversation receives append, prepend, replace, or Assistant settlement from the existing feed. +2. The assembler runs existing Node Definition matching, state updates, and required Context replay. +3. Existing immediate/animation-frame/none cadence schedules publication; grouping adds no timer or event subscription. +4. Flush materializes Step then Turn Location data and target Nodes. +5. Builder.replace/apply installs projected Nodes, target order, and indexes, records position changes, and retains GroupInput with its indexed readers without notifying independent sources. +6. The assembler obtains the target's registered Group Definition context, calls update with builder.groupInput(), and adopts the returned State. +7. It calls buildGroups, validates and installs the result, and installs the target snapshot. +8. After every affected target is installed, it calls Builder.publish, publishes Group sources, and publishes Location data. +9. BoundConversation publishes the existing root Conversation source. + +First activation's replaceView uses the same target update function as ordinary flush. Inactive targets create neither Builder nor Group context. Registration rebuild takes the View and Group Definitions together, discards replaced/removed Group contexts, and clears old observed output. Surviving Definitions reuse keyed stores. React receives the Node Store identity so replacement rebinds keyed hooks without replacing keys or DOM instances. + +Removing a registered View Definition pauses its grouping and clears derived output, while retaining the Group Definition registration. Restoring the View uses the existing replacement flow to rebuild from the current loaded timeline. Initial Group registration still rejects a missing View target; switching tabs does not unregister a View. + +Grouping never replays raw events. Node Definitions handle replay first; grouping consumes the resulting target replacement or changes. Prepend may repair boundaries and legitimately move members. Definition handles old/new Locations and changed Turns instead of treating pagination or every content chunk as a complete-history regroup. + +## Concrete implementation locations + +| Layer/file | Responsibility | +|---|---| +| [groups.ts](../../../../packages/client/ui-conversation/src/client/contract/groups.ts) | Group Definition, input, references, updates, typed data map, and readers. | +| [conversation.ts](../../../../packages/client/ui-conversation/src/client/contract/conversation.ts) | Optional Builder groupInput/publish and snapshot-store grouped reader; existing Node/View Definitions stay unchanged. | +| [group-registry.ts](../../../../packages/client/ui-conversation/src/client/conversation/group-registry.ts) | Existing registry base, target uniqueness, typed registration, effect/disposer lifecycle. | +| [assembly.ts](../../../../packages/client/ui-conversation/src/client/conversation/assembly.ts) | UiConversation.groups and existing binding rebuild notifications. | +| [assembler.ts](../../../../packages/client/ui-conversation/src/client/conversation/assembler.ts) | Common first-activation/flush dispatch, context ownership, installation, and publication. | +| [location-index.ts](../../../../packages/client/ui-conversation/src/client/conversation/location-index.ts) | Accumulate changed Turns, including lifecycle-only and Location-data changes. | +| [group-store.ts](../../../../packages/client/ui-conversation/src/client/conversation/group-store.ts) | Atomic reference validation, keyed sources, array reuse, and local publication. | +| [chat-snapshot-builder.ts](../../../../packages/client/ui-chat/src/client/conversation-nodes/chat-snapshot-builder.ts) | Record projected Node deltas, target positions, and changed Turn orders; provide indexed readers and defer source notification. No process grouping class. | +| [ChatView.tsx](../../../../packages/client/ui-chat/src/client/chat/ChatView.tsx) | Read optional root entries and switch between node/group; fallback to existing Node order. | +| [ChatGroupSeat.tsx](../../../../packages/client/ui-chat/src/client/chat/ChatGroupSeat.tsx) | Stable group parent, member-only subscription, nested Node seats. | +| [ChatNodeSeat.tsx](../../../../packages/client/ui-chat/src/client/chat/ChatNodeSeat.tsx) | Existing Node sources/renderers, groupPart forwarding, distinct part anchors, Store-replacement rebind. | +| [slots.ts](../../../../packages/client/ui-chat/src/client/contract/slots.ts) and [apply.ts](../../../../packages/client/ui-chat/src/client/apply.ts) | Existing keyed-hook injection for Group sources; no Slot-engine change. | + +## React reading and presentation + +The root selects `views.grouped('chat')?.entries` through the existing useConversation source. Keyed Group hooks use existing injection; each Group seat selects members. Members use existing keyed Node sources. Removed Group sources can read undefined before the root removes their components, so selectors tolerate absence and keep Hook order stable. + +React keys derive from reference kind, NodeKey/groupPart or GroupKey, using unambiguous tuples. No separate RenderKey type or renderer dispatch field is needed. Compact, Detailed, and Expanded keep the same group container and member parents; presentation changes do not rerun the Group Definition or select a different root branch. + +The stable Group parent is a `div` with `display: contents`, without its own layout box. CSS inheritance remains available; business styles adapt child/sibling selectors and own measurable body containers. Neither CSS variables nor geometry enter the Definition. Reading-position capture measures member Nodes and retains part-specific anchors; existing Turn navigation resolves the original NodeKey to its first visible part. + +## Alternatives considered + +**Builder-owned buildGroups or process projector.** Rejected: the target aggregator would still own business rules. Moving caches into infrastructure does not change that ownership. + +**Add buildGroups to the existing event Definition without new input.** Rejected: the callback name does not provide cross-Node input, lifecycle changes, or reconstruction semantics. + +**Replay raw events in the Group Definition.** Rejected: it duplicates Assistant/Tool/Message interpretation and can diverge from the Builder's final visible order. + +**ConversationViewDefinition.buildLayout.** Rejected: it broadens a target factory into a business layout callback rather than registering a distinct business Definition. + +**One event Context per Group.** Rejected: boundaries depend on preceding visible content, not a standalone event carrying the group's start identity. A target-local derived context also admits independent Nodes outside Turns. + +**Dependency graphs, ViewItem classes, factories, nested groups, or strategy stacking.** Rejected for this phase: one target input and one non-nested grouping output cover the stated need. + +**Separate data-only mutation operation.** Rejected: an immutable snapshot upsert with reused members already updates summaries without replacing other groups. + +**Let Group Definition add/remove both Node and Group entities.** Rejected: Node data already has an owner. Symmetrical references do not require a second Node owner. + +**Remove the group wrapper in Expanded.** Rejected: changing the parent remounts members even when their keys survive. + +## Verification + +- [Group store tests](../../../../packages/client/ui-conversation/tests/conversation-group-store.client.spec.ts) cover atomic reference validation, root/member identity reuse, local publication, and removal without deleting source Nodes. +- [Grouping dispatch tests](../../../../packages/client/ui-conversation/tests/conversation-groups.client.spec.ts) cover first activation, lifecycle-only input, registry replacement, View removal/recovery, and complete replacement output. [Assembler tests](../../../../packages/client/ui-conversation/tests/conversation-assembler.client.spec.ts) cover changed-Turn reporting and Location data sources. +- [Node source tests](../../../../packages/client/ui-chat/tests/chat-node-source.client.spec.ts) cover projected grouping inputs, indexed readers, and empty change batches. +- [Chat rendering tests](../../../../packages/client/ui-chat/tests/chat-view.client.spec.tsx) retain component state across modes, rebind replacement Node stores, pass independent parts, and omit unreferenced Nodes. [Viewport tests](../../../../packages/client/ui-chat/tests/chat-viewport.client.spec.ts) cover grouped reading anchors, history prepend, and part-aware Turn navigation. + +## Consequences + +- This is an explicit framework extension: a new input protocol, registry, context, publication phase, and reader. It is not merely another callback. +- Target-local State does not automatically make business updates local. Node changes and changed Turn orders let a Definition choose affected groups and ranges. Structural changes still rebuild the target position index over visible order; content-only updates do not rebuild it. Structural grouping output still replaces the complete root-reference array. +- Node order and visibility have one owner: the Builder. Grouping must not independently sort raw events or infer membership again in infrastructure. +- Stable mounting does not eliminate layout, paint, or retained-memory costs. No measured latency or optimal-performance claim is made. +- Real group splits, merges, first-member changes, and pagination repairs can change identity. The mode-switch guarantee does not prohibit those legitimate changes. +- Revealing hidden parts, selection/copy, interruption notices, group disclosure, and outer Turn interaction require business adaptation. Foundation tests exercise mixed references without enabling production groups. diff --git a/.agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.zh.md b/.agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.zh.md new file mode 100644 index 0000000000..70657ded8e --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-09-21-conversation-build-groups.zh.md @@ -0,0 +1,155 @@ +# Agent Note: Definition 拥有分组业务，框架统一调度 Node／Group 输出 + +Status: implemented + +[English](2026-09-21-conversation-build-groups.md) | 中文 + +## Problem + +Chat 需要把连续过程内容收进可展开的组，同时保留独立回复、用户输入和轮次控件。Compact、Detailed、Expanded 必须共用分组结果；切换模式不能通过删除组容器、移动成员父级或更换 key 重建 React 实例。 + +Chat Builder 汇总整个目标的节点及索引。在其中固定创建 Chat 过程投影器，即使把部分缓存移到通用层，仍然让 Builder 拥有业务分段。业务 Definition 才是这些规则的注册点。 + +过程组与 Step 正交：同一个 Assistant Node 可以把推理放在组内、回复放在组外，回复后的工具进入后面的组。Step 编号范围无法描述这种成员归属。现有事件 Definition 逐个解释事件，分组则需要这些 Definition 已经解释完成的节点。 + +## Decision + +分组基础机制提供独立的节点输入 `ConversationGroupDefinition`、注册表及 Session 内上下文、Builder 通用输入与发布、按键 Group 存储及两种 React 分支。基础机制不注册真实 Chat 分段及 PR #4565 展示，[业务迁移提案](../../proposed/feature/2026-09-20-chat-work-details-migration.zh.md)保留这些需求。[子系统参考](../../../../docs/subsystems/conversation.zh.md#group-definitions)与[源码类型](../../../../packages/client/ui-conversation/src/client/contract/groups.ts)拥有当前 API 细节。 + +### 职责 + +| 要求 | 含义 | +|---|---| +| 分组业务属于已注册 Definition | 成员、分段、回复、steering、重试、摘要和缓存失效放在一起。 | +| Builder 与 assembler 做统一工作 | 提供输入、调度 Definition、校验引用、复用身份并发布，不创建具体分组类。 | +| 产品行为以 PR #4565 为准 | 保留行为，不复制其侵入性实现。 | +| React 根循环只有两种 kind | 根顺序包含 NodeReference 和 GroupReference，不引入另一套布局对象模型。 | +| 所有模式共用分组 | 模式不进入 Definition 输入、key 或父级选择。 | +| 不采用 buildLayout | ConversationViewDefinition 保留现有职责。 | +| 先基础、后业务 | 基础 PR 不包含真实 Chat 分组、摘要、通知或页脚变化。 | + +### 与已有记录的关系 + +| 记录 | 关系 | +|---|---| +| [业务节点组装](2026-08-09-client-conversation-node-assembly.zh.md) | 保留事件匹配、Context、Location、每个 Context 一个业务 Node 及目标 Builder，分组增加另一种输入类别。 | +| [展示策略与过程分组](../../proposed/architecture/2026-09-20-chat-presentation-policy-and-step-groups.zh.md) | 保留 Definition 归属、局部订阅和模式无关要求，以新分组机制替代事件 fold 及 Step 范围成员推断。 | +| [工作过程展示迁移](../../proposed/feature/2026-09-20-chat-work-details-migration.zh.md) | 保留产品范围及迁移跟踪，基础机制本身不代表这些功能完成。 | + +这些记录仍有独立理由，保留有效状态。跨 View 导航与 `toolCallFocus` 不属于本次改动；不引入 View 句柄、Label Slot 或资源导航架构。 + +## Definition 输入、状态与注册 + +`ConversationNodeDefinition` 保留 `match/start/update/publication/buildLocationData/buildViewNode`，`ConversationViewDefinition` 保留 `target/create/isActive/toolCallFocus`。分组业务通过 `ctx.uiConversation.groups.register(processGroupDefinition)` 注册，不加在上述已有 Definition 的回调中。 + +- 每个目标只有一个 Group Definition，拥有完整分组语义。重复目标注册报错，不选择胜者，也不叠加策略。 +- 注册要求已有 View 目标，但不构造 Builder。首次激活取得 Builder 后检查 `groupInput()`，缺少输入方法时报错。未分组目标不要求该方法，也不分配 Group 状态。 +- 每个 Session 为已注册 Definition 拥有一个派生上下文。`create()` 初始化 State，不是 `turn/start` 事件，也不是事件 Definition 的唯一 start Match。Definition 对象不保存跨 Session 的可变状态。 +- `update(context, input)` 返回框架采纳的 State。`buildGroups(context)` 物化待输出结果，不重新解释事件，不因重复读取增加计数。替换输入要求输出完整根序列和组替换，避免已移除节点留下悬挂引用；apply 的 `null` 保留结果。 +- `replace` 提供目标顺序、时间线及同步的 `readNode`、`readTurn`、`readPosition` 读取器；`apply` 还提供投影后的节点前后值、`changedTurns` 和 `changedTurnOrders`。读取器仅在当前同步调用中有效，不保留到 State。 +- 节点数据仍在目标 Node Store。仅正文更新保留顺序数组；旧值在 Builder 安装投影结果前记录，也覆盖目标投影额外改变的节点。 +- `ConversationLocationIndex` 从边界事件和 Location 数据写入累计变化轮次。assembler 将同一批变化交给全部被更新目标，覆盖没有节点 upsert 的 Turn 结束。直接调用未分组 Builder 的调用方可省略该字段。 +- 目标位置索引通过 `changedTurnOrders` 提供可见键、所属 Turn 或紧邻关系的变化，包含移动 Node 的新旧所属轮次，以及受相邻轮次外 Node 影响的轮次。`readTurn` 返回某个 Turn 的可见键，`readPosition` 返回包含所属 Turn 和前后紧邻键的 `GroupNodePosition`。仅按 Turn 读取会遗漏轮次外 Node 造成的分隔，业务分段可以通过相邻事实保留这些分隔。索引提供位置，不决定分组。 +- 通用 GroupDataMap 在注册、存储及读取之间关联目标和 Data。未声明的目标没有组载荷类型，业务消费方不通过 unknown 断言恢复摘要类型。 + +## Node／Group 引用与更新 + +`NodeKey` 表示既有 Context Node，`GroupKey` 在 Session、目标及 Definition 内标识组。`NodeReference` 与 `GroupReference` 在 `RenderEntry` 中地位相同。`GroupSnapshot` 单独保存组的不可变数据与有序 NodeReference 成员，不混入根顺序。创建 Group 不增加 Session 事件或执行生命周期。 + +`groupPart` 是渲染器拥有的节点部分，例如 reasoning 或 response，省略表示整个 Node。部分引用也可以独立位于根层。部分内容的完备性和归属由业务渲染器负责，不由框架判断。组数据描述状态、计数或摘要，不复制 Node 正文、工具结果或可变业务 State。组不携带 renderer 字段、CSS、坐标、React 元素、导航或嵌套组。 + +| 更新 | 含义 | +|---|---| +| apply 输入省略 `entries` | 保留根顺序；替换输入必须提供完整序列。 | +| 提供 `entries`，包括 `[]` | 替换完整根序列，空数组清空根序列。 | +| `groups: { kind: 'replace', snapshots }` | 替换完整组记录，移除未包含的旧组。 | +| `groups: { kind: 'apply', upserts, removes }` | upsert 完整快照，只按具名 GroupKey 删除组。 | +| 仅数据变化 | upsert 新 data，复用 members，省略 entries。 | +| 仅成员变化 | upsert 该组完整的新成员数组，不替换全部组。 | +| 仅根顺序变化 | 提供 entries，apply 的 upserts/removes 为空。 | +| 解散一个组 | 原子地用成员替换根引用并删除 GroupKey，保留原 Node。 | + +`entries` 与 `groups` 属于同一安装批次，不是独立发布通道。删除成员引用不删除 Node。upsert 需要快照内容，删除只需要身份。真实成员归属变化可以让 React 重挂载成员，但模式切换不能解散组。 + +存储在修改前校验最终提交结果：每个根 Group 恰有一份记录和一个根位置；Node 引用存在；重复 upsert、重复删除及同时 upsert/删除均报错。全部根和成员中，每个 `(NodeKey, groupPart)` 最多出现一次。整 Node 不能与自身任一部分共存，不同部分可以占据不同位置。允许引用在一次原子更新中移动位置。 + +GroupStore 按 GroupKey 索引记录和来源，不反复查找数组。等价根数组、成员数组及未变化组快照保留身份。仅数据 upsert 只检查提交的快照，不扫描根、未变化成员或 Node。完整替换重新校验引用，仍按 key 对齐结果。Definition 缓存拥有业务计算，GroupStore 拥有已安装结果及通知。 + +## 组装时序与生命周期 + +1. BoundConversation 从现有 feed 接收 append、prepend、replace 或 Assistant settlement。 +2. assembler 运行既有 Node Definition 匹配、状态更新及必要的 Context 重放。 +3. 既有 immediate/animation-frame/none 节奏调度发布，分组不增加计时器或事件订阅。 +4. flush 依次物化 Step、Turn Location 数据及目标 Node。 +5. Builder.replace/apply 安装投影后 Node、目标顺序及索引，记录位置变化，保留包含索引读取器的 GroupInput，不通知独立来源。 +6. assembler 取得目标已注册的 Group Definition 上下文，以 builder.groupInput() 调用 update，采纳返回 State。 +7. 调用 buildGroups，校验并安装结果，安装目标快照。 +8. 全部受影响目标安装完成后，调用 Builder.publish，发布 Group 来源和 Location 数据。 +9. BoundConversation 发布既有根 Conversation 来源。 + +首次激活的 replaceView 与普通 flush 使用同一目标更新函数。未激活目标不创建 Builder 或 Group 上下文。注册重建一并采纳 View 和 Group Definition，丢弃被替换或删除的 Group 上下文并清空旧观察结果。未替换的 Definition 复用按键存储。React 接收 Node Store 身份，让存储替换时重新绑定按键钩子，不替换 key 或 DOM 实例。 + +已注册的 View Definition 被移除后，分组计算暂停并清空派生结果，Group Definition 注册仍保留。View 恢复时沿现有替换流程，从当前已加载时间线重建。首次注册 Group 时仍拒绝缺失的 View 目标；切换页签不会注销 View。 + +分组不重放原始事件。Node Definition 先处理重放，分组消费其最终目标替换或变化。prepend 可以修正边界并真实移动成员。Definition 处理新旧 Location 和变化轮次，不把分页或每个正文分片都当作完整历史重新分组。 + +## 具体实现落点 + +| 层／文件 | 职责 | +|---|---| +| [groups.ts](../../../../packages/client/ui-conversation/src/client/contract/groups.ts) | Group Definition、输入、引用、更新、类型数据映射及读取器。 | +| [conversation.ts](../../../../packages/client/ui-conversation/src/client/contract/conversation.ts) | Builder 可选 groupInput/publish 和快照存储 grouped 读取，已有 Node/View Definition 不变。 | +| [group-registry.ts](../../../../packages/client/ui-conversation/src/client/conversation/group-registry.ts) | 复用注册表基类，目标唯一性、类型化注册及 effect/disposer 生命周期。 | +| [assembly.ts](../../../../packages/client/ui-conversation/src/client/conversation/assembly.ts) | UiConversation.groups 与现有 binding 重建通知。 | +| [assembler.ts](../../../../packages/client/ui-conversation/src/client/conversation/assembler.ts) | 首次激活和 flush 共用调度，上下文归属、安装及发布。 | +| [location-index.ts](../../../../packages/client/ui-conversation/src/client/conversation/location-index.ts) | 累积变化轮次，覆盖仅生命周期和 Location 数据变化。 | +| [group-store.ts](../../../../packages/client/ui-conversation/src/client/conversation/group-store.ts) | 原子引用校验、按键来源、数组复用及局部发布。 | +| [chat-snapshot-builder.ts](../../../../packages/client/ui-chat/src/client/conversation-nodes/chat-snapshot-builder.ts) | 记录投影后节点增量、目标位置及变化轮次顺序，提供索引读取器并推迟来源通知，不持有过程分组类。 | +| [ChatView.tsx](../../../../packages/client/ui-chat/src/client/chat/ChatView.tsx) | 读取可选根引用并切换 node/group，无分组时使用既有 Node 顺序。 | +| [ChatGroupSeat.tsx](../../../../packages/client/ui-chat/src/client/chat/ChatGroupSeat.tsx) | 稳定组父级、仅成员订阅及嵌套 Node 容器。 | +| [ChatNodeSeat.tsx](../../../../packages/client/ui-chat/src/client/chat/ChatNodeSeat.tsx) | 既有 Node 来源及渲染器、groupPart 传递、独立部分锚点及 Store 替换时重绑定。 | +| [slots.ts](../../../../packages/client/ui-chat/src/client/contract/slots.ts) 与 [apply.ts](../../../../packages/client/ui-chat/src/client/apply.ts) | 使用现有按键钩子注入 Group 来源，不改 Slot 引擎。 | + +## React 读取与展示 + +根层通过既有 useConversation 来源选择 `views.grouped('chat')?.entries`。按键 Group 钩子沿用现有注入，Group 容器选择 members。成员使用既有按键 Node 来源。根列表移除组件前，已删除 Group 来源可能先返回 undefined，因此选择器允许不存在，并保持钩子顺序稳定。 + +React key 由引用 kind、NodeKey/groupPart 或 GroupKey 的无歧义元组派生，不另设 RenderKey 类型或 renderer 分发字段。Compact、Detailed、Expanded 保留同一组容器及成员父级，展示变化不重新运行 Group Definition，也不选择不同的根分支。 + +稳定 Group 父级使用 `div` 与 `display: contents`，自身不产生布局盒子。CSS 继承仍然可用，业务样式适配子级／兄弟选择器并拥有可测量的正文容器。CSS 变量和几何信息都不进入 Definition。阅读位置采样测量成员 Node，保留部分专属锚点；既有轮次导航将原 NodeKey 解析到它的第一个可见部分。 + +## Alternatives considered + +**Builder 持有 buildGroups 或过程投影器。** 不采用，目标汇总器仍然拥有业务规则，把缓存移到通用层不能改变归属。 + +**在原事件 Definition 加 buildGroups，不补输入。** 不采用，回调名称不能提供跨节点输入、生命周期变化或重建语义。 + +**Group Definition 重放原始事件。** 不采用，它重复 Assistant、Tool、Message 解释，并可能偏离 Builder 最终可见顺序。 + +**ConversationViewDefinition.buildLayout。** 不采用，它把目标工厂扩展成业务布局回调，没有独立注册业务 Definition。 + +**每个 Group 一个事件 Context。** 不采用，边界取决于前方可见内容，不由一个携带组起点身份的事件决定。目标内派生上下文也能处理 Turn 外独立节点。 + +**依赖图、ViewItem 类、工厂、嵌套组或策略叠加。** 本阶段不采用，一个目标输入和一份非嵌套分组输出覆盖当前诉求。 + +**独立的数据更新操作。** 不采用，复用 members 的不可变快照 upsert 已经能更新摘要，不替换其他组。 + +**Group Definition 同时增删 Node 和 Group 实体。** 不采用，Node 数据已有所有者，引用对称不需要第二个 Node 所有者。 + +**Expanded 移除组容器。** 不采用，即使 key 不变，父级改变仍会重挂载成员。 + +## Verification + +- [Group 存储测试](../../../../packages/client/ui-conversation/tests/conversation-group-store.client.spec.ts)覆盖原子引用校验、根与成员身份复用、局部发布，以及删除组但保留原 Node。 +- [分组调度测试](../../../../packages/client/ui-conversation/tests/conversation-groups.client.spec.ts)覆盖首次激活、仅生命周期输入、注册替换、View 移除与恢复，以及完整替换输出。[Assembler 测试](../../../../packages/client/ui-conversation/tests/conversation-assembler.client.spec.ts)覆盖变化轮次报告和 Location 数据来源。 +- [Node 来源测试](../../../../packages/client/ui-chat/tests/chat-node-source.client.spec.ts)覆盖投影后的分组输入、索引读取器和空变化批次。 +- [Chat 渲染测试](../../../../packages/client/ui-chat/tests/chat-view.client.spec.tsx)保留模式切换时的组件状态，重绑定替换后的 Node 存储，传递独立部分，并省略未引用 Node。[视口测试](../../../../packages/client/ui-chat/tests/chat-viewport.client.spec.ts)覆盖组内阅读锚点、历史前插及部分感知的轮次导航。 + +## Consequences + +- 这是明确的框架扩展，包含新输入协议、注册表、上下文、发布阶段及读取器，不只是增加一个回调。 +- 目标内 State 不会自动让业务更新局部化，Definition 可以根据节点变化和轮次顺序变化选择受影响的组及范围。结构变化仍按可见顺序重建目标位置索引，仅正文更新不重建该索引。结构性分组输出仍替换完整根引用数组。 +- Node 顺序及可见性只有 Builder 一个所有者，分组不独立排序原始事件，也不在通用层再次推断成员。 +- 稳定挂载不消除布局、绘制或保留内存的成本，不宣称实测延迟或最优性能。 +- 真实组拆并、首成员变化及分页修正可以改变身份，模式切换保证不禁止这些正常变化。 +- 隐藏部分的揭示、选择复制、中断提示、组开合与外层 Turn 联动需要业务适配。基础测试使用混合引用，不启用真实分组。 diff --git a/.agents/notes/implemented/feature/2026-09-16-browser-excel-preview.i18n.yaml b/.agents/notes/implemented/feature/2026-09-16-browser-excel-preview.i18n.yaml new file mode 100644 index 0000000000..f279a70f08 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-09-16-browser-excel-preview.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-09-16-browser-excel-preview.md +2026-09-16-browser-excel-preview.md: 5631906931122e45d4381a570c095203655b6435 +2026-09-16-browser-excel-preview.zh.md: 4107b986246415bc640b683650d2b18f03a786c8 diff --git a/.agents/notes/implemented/feature/2026-09-16-browser-excel-preview.md b/.agents/notes/implemented/feature/2026-09-16-browser-excel-preview.md new file mode 100644 index 0000000000..5631906931 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-09-16-browser-excel-preview.md @@ -0,0 +1,35 @@ +# Agent Note: Browser Excel preview with saved formula results + +Status: implemented + +English | [中文](2026-09-16-browser-excel-preview.zh.md) + +## Problem + +PDF conversion loses spreadsheet navigation and the relationship between a formula and its saved value. Spreadsheet preview also needlessly depends on the Host Office engine when the browser can read the workbook. + +## Decision + +The [document preview](../../../../packages/client/ui-sidebar-documentpreview/README.md#excel-preview) opens XLSX, XLS, CSV, and TSV with FortuneSheet and package-local adapters for ExcelJS, SheetJS CE, and PapaParse. The adapter accepts bytes, format, and limits and returns worksheet data; it has no React state setters, timers, or component references. Dependencies own file parsing, while the adapters share display formatting and initial selection without introducing another workbook model. CSV/TSV retain literal strings and a plain-text option; implicit type inference could otherwise discard leading zeros or interpret source text as formulas. The shared document reader calls `remote.workspaceFiles.readBytes(sessionId, path, {}, signal)` and receives native bytes through the binary Remote, retaining Session authorization, file versions, reload, and transient bytes without base64 decoding. Workspace Files owns bounded file reads; format parsing stays in the Client. The registration marks only XLSX/XLS as `binaryExtensions`, so the shared document container offers Plain text for CSV/TSV. Each renderer registers its body under its own id. + +Preview is read-only and retains formula text alongside saved results. It does not recalculate, so unsupported functions cannot replace a valid saved result with an error. A formula without a saved result remains blank. Workbooks containing formulas show a compact warning beside the formula icon instead of consuming worksheet height with a notice row. The spreadsheet is neither an authoring tool nor a full Excel rendering engine. + +A disposable browser Worker contains parser CPU work. A source-byte limit precedes parsing, a combined rectangular-area limit precedes FortuneSheet matrix allocation, and a timeout terminates stalled parsing. These limits do not claim a hard memory sandbox. Styles are scoped to the renderer; external hyperlinks are text only. + +The pinned FortuneSheet React dependency carries a [patch](../../../../patches/@fortune-sheet__react@1.0.4.patch) that skips the input box's read-only layout state update unless formula-reference highlighting is enabled. The unconditional update otherwise causes a maximum-update-depth failure when switching worksheets under React 19. Sheet activation also retains the adapter's complete initial selection instead of clearing it and leaving an empty or invalid address. The browser scenario covers these failures through the bundled ESM entry. + +The read-only formula bar takes formulas from raw `cell.f` and writes formulas and cell text through `textContent`. FortuneSheet's formula formatter emits HTML spans containing document text, and its HTML helper passes formulas and strings starting with `