From afe85c1cfdadbf95bf6aeef5ccfc224c0940cff2 Mon Sep 17 00:00:00 2001 From: yudshj Date: Thu, 17 Sep 2026 14:43:30 +0800 Subject: [PATCH] fix(preview): preserve PDF text layer rotation --- ...9-08-document-preview-operations.i18n.yaml | 4 +-- .../2026-09-08-document-preview-operations.md | 2 +- ...26-09-08-document-preview-operations.zh.md | 2 +- apps/web/tests/document-preview.e2e.ts | 30 +++++++++++++++++++ .../README.i18n.yaml | 4 +-- .../ui-sidebar-documentpreview/README.md | 2 +- .../ui-sidebar-documentpreview/README.zh.md | 2 +- .../src/client/pdf/PdfBody.module.css | 12 ++++++++ .../src/client/pdf/text.ts | 3 +- .../tests/pdf-fixture.ts | 7 +++-- .../tests/pdf-text.client.spec.ts | 4 +-- .../web/document-preview/document.expected.md | 4 +++ 12 files changed, 62 insertions(+), 14 deletions(-) diff --git a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.i18n.yaml b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.i18n.yaml index 86e96590ef..60e03dbff7 100644 --- a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md -2026-09-08-document-preview-operations.md: ea315641728cefc69bb67be5495e93cbaa39f1fe -2026-09-08-document-preview-operations.zh.md: 71dfb0f0bdccc355cf0edeb33ce56a72de2974a7 +2026-09-08-document-preview-operations.md: 980e2e443db13d6a956d554172bc765a5c11b435 +2026-09-08-document-preview-operations.zh.md: 364ef9ced6f8d98313f9bc287588c98daa408837 diff --git a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md index ea31564172..980e2e443d 100644 --- a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md +++ b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.md @@ -20,7 +20,7 @@ Readable files use `dsh-resource://file/session//`. The path ma Markdown and code reuse the incremental primitives with cumulative paged text. HTML, PDF, and images read complete `Uint8Array` data; Host transport remains base64. Published buffers are borrowed read-only and never persist into layout or Session JSON. PDF.js runs in an owned Worker with version-matched bundled font and decoder data, and copies input before transfer to preserve Preview's retained buffer. HTML runs in a Blob iframe with `sandbox="allow-scripts"`, without same-origin, popup, form, download, or top-navigation privileges. The browser retains its normal external-network rules. Bounded static local JS/CSS reads stay in the parent; the opaque frame creates its own asset Blobs, because it cannot load parent-origin Blobs. PNG, JPEG, GIF, WebP, BMP, ICO, and SVG use image-specific Blob URLs in an `` static-image context. An image wider than the pane scales down to its width at its aspect ratio; a smaller image keeps its intrinsic CSS-pixel dimensions centred by auto margins, and a taller image extends the shared scroller's vertical range ([sidebar preview polish](../feature/2026-09-11-sidebar-document-preview-polish.md)). The renderer provides no zoom or drag-to-pan. SVG markup never enters the application DOM or an iframe, so scripts remain inert and cannot reach the parent page. Replacing HTML or an image revokes its root Blob URL. -PDF.js's official TextLayerBuilder owns selection boundaries and copy normalization over the width-fitted canvas, with shared page cleanup and a component-owned resize observer. Its end-of-content marker and stacking rules constrain selection in blank regions; line-break highlighting is suppressed. Per-page cancellation uses the builder's cleanup rather than aborting the first page's signal, because the official selection listeners are shared across pages. +PDF.js's official TextLayerBuilder owns selection boundaries and copy normalization over the width-fitted canvas, with shared page cleanup and a component-owned resize observer. Responsive sizing uses the CSS `scale` property independently of PDF.js's page rotation and translation transforms. Its end-of-content marker and stacking rules constrain selection in blank regions; line-break highlighting is suppressed. Per-page cancellation uses the builder's cleanup rather than aborting the first page's signal, because the official selection listeners are shared across pages. ## Alternatives considered diff --git a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.zh.md b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.zh.md index 71dfb0f0bd..364ef9ced6 100644 --- a/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.zh.md +++ b/.agents/notes/implemented/architecture/2026-09-08-document-preview-operations.zh.md @@ -20,7 +20,7 @@ Document Preview 将资源观察与内容读取分开。[资源模型](2026-09-0 Markdown 和代码通过累积的分页文本复用增量渲染原语。HTML、PDF 和图片读取完整 `Uint8Array` 数据；Host 传输保持 base64。发布后的缓冲区只读借用，绝不持久化进布局或 Session JSON。PDF.js 在自有 Worker 中运行，字体和解码数据以相同版本随包发布，转移输入前先复制，以保留 Preview 的缓冲区。HTML 在 Blob iframe 中运行，设置 `sandbox="allow-scripts"`，不授予同源、弹窗、表单、下载或顶层导航权限。浏览器保持正常的外部网络规则。有上限的静态本地 JS/CSS 读取由父页面负责；不透明源 iframe 创建自己的资源 Blob，因为它不能加载父源创建的 Blob。PNG、JPEG、GIF、WebP、BMP、ICO 和 SVG 使用图片专用 Blob URL，在 `` 静态图片上下文中渲染。比面板宽的图片按纵横比缩小到面板宽度；较小的图片保留固有 CSS 像素尺寸并由 auto margin 居中，较高的图片扩展共享滚动区的纵向范围（[侧边栏预览打磨](../feature/2026-09-11-sidebar-document-preview-polish.zh.md)）。渲染器不提供缩放或拖拽平移。SVG 标记绝不进入应用 DOM 或 iframe，因此脚本保持不可执行，也无法访问父页面。替换 HTML 或图片时会撤销其根 Blob URL。 -PDF.js 官方 TextLayerBuilder 在适配宽度的 canvas 上负责选择边界和复制规范化，共享页面清理，并使用由组件拥有的 resize observer。其内容结束标记和堆叠规则限制空白区域中的选择；换行高亮被抑制。逐页取消使用 builder 的清理操作，而不 abort 第一页的信号，因为官方选择监听器跨页面共享。 +PDF.js 官方 TextLayerBuilder 在适配宽度的 canvas 上负责选择边界和复制规范化，共享页面清理，并使用由组件拥有的 resize observer。响应式尺寸适配使用独立的 CSS `scale` 属性，与 PDF.js 的页面旋转和平移变换组合。其内容结束标记和堆叠规则限制空白区域中的选择；换行高亮被抑制。逐页取消使用 builder 的清理操作，而不 abort 第一页的信号，因为官方选择监听器跨页面共享。 ## 考虑过的替代方案 diff --git a/apps/web/tests/document-preview.e2e.ts b/apps/web/tests/document-preview.e2e.ts index ff95742f46..24137b06ad 100644 --- a/apps/web/tests/document-preview.e2e.ts +++ b/apps/web/tests/document-preview.e2e.ts @@ -157,6 +157,7 @@ describe.skipIf(MODE === 'record')('web e2e: document preview through Files', () ].join('')), writeFile(join(cwd, 'smoke.pdf'), pdfFixture()), writeFile(join(cwd, 'user-unit.pdf'), pdfFixture(2)), + ...[90, 180, 270].map(rotation => writeFile(join(cwd, `rotated-${rotation}.pdf`), pdfFixture(4, rotation))), writeFile(join(cwd, 'selection.pdf'), selectionPdfFixture()), ...['doc', 'docx', 'xls', 'xlsx', 'ppt', 'pptx'].map(extension => writeFile(join(cwd, `unavailable.${extension}`), Buffer.from('PK\u0003\u0004OFFICE_BINARY_PREVIEW'))), writeFile(join(cwd, 'clip.mp4'), Buffer.from([0x00, 0x00, 0x00, 0x18, 0x66, 0x74, 0x79, 0x70])), @@ -349,6 +350,35 @@ describe.skipIf(MODE === 'record')('web e2e: document preview through Files', () await copyPdfText(page, preview, 'Selectable PDF text') sections.push('## PDF page units\n\n- UserUnit 2: selected and copied text aligns with the canvas') + const viewportSize = page.viewportSize()! + try { + for (const rotation of [90, 180, 270]) { + await openFile(`rotated-${rotation}.pdf`) + for (const width of [viewportSize.width, 1280]) { + await page.setViewportSize({ ...viewportSize, width }) + await copyPdfText(page, preview, 'Selectable PDF text') + // The fixture's only black pixels are text; canvas ink is independent of the overlay geometry. + await expect.poll(() => preview.locator('[data-pdf-page]').first().evaluate((node) => { + const canvas = node.querySelector('canvas')! + const canvasBox = canvas.getBoundingClientRect() + const textBox = node.querySelector('.textLayer span')!.getBoundingClientRect() + const pixels = canvas.getContext('2d')!.getImageData(0, 0, canvas.width, canvas.height).data + let ink = 0 + let aligned = 0 + for (let i = 0; i < pixels.length; i += 4) { + if (pixels[i + 3]! < 128 || Math.max(pixels[i]!, pixels[i + 1]!, pixels[i + 2]!) > 80) continue + ink++ + const x = canvasBox.left + ((i / 4) % canvas.width + 0.5) * canvasBox.width / canvas.width + const y = canvasBox.top + (Math.floor(i / 4 / canvas.width) + 0.5) * canvasBox.height / canvas.height + if (x >= textBox.left - 1 && x <= textBox.right + 1 && y >= textBox.top - 1 && y <= textBox.bottom + 1) aligned++ + } + return ink === 0 ? 0 : aligned / ink + })).toBeGreaterThan(0.95) + } + } + } finally { await page.setViewportSize(viewportSize) } + sections.push('## PDF page rotation\n\n- 90, 180, 270 degrees: selection and copied text align with canvas ink before and after resizing') + await openFile('selection.pdf') await preview.getByRole('img', { name: 'PDF page 1', exact: true }).waitFor({ state: 'visible' }) const selectionLayer = preview.locator('.textLayer') diff --git a/packages/client/ui-sidebar-documentpreview/README.i18n.yaml b/packages/client/ui-sidebar-documentpreview/README.i18n.yaml index fce4f209cc..7d9c54945e 100644 --- a/packages/client/ui-sidebar-documentpreview/README.i18n.yaml +++ b/packages/client/ui-sidebar-documentpreview/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/client/ui-sidebar-documentpreview/README.md -README.md: 3c5eeb63beb3e3cab329437ffea69a8da15a3b4d -README.zh.md: 4d616fe1924dce6949d1dd78be0479e11229839e +README.md: 3b8b2b1d98e45caadb646a43fd04c74d600d0cd3 +README.zh.md: 94de2f75e5955e5bb293d398e65f615b16ece4e4 diff --git a/packages/client/ui-sidebar-documentpreview/README.md b/packages/client/ui-sidebar-documentpreview/README.md index 3c5eeb63be..3b8b2b1d98 100644 --- a/packages/client/ui-sidebar-documentpreview/README.md +++ b/packages/client/ui-sidebar-documentpreview/README.md @@ -56,7 +56,7 @@ PNG, JPEG, GIF, WebP, BMP, ICO, and SVG render through Blob URLs in an `` s Shared copy comes from `sidebarDocumentPreview`; each builtin renderer owns its localized labels. PDF and converted Office previews use a graphite background in light mode and a matte-black background in dark mode, with subtle page shadows and original document colors. -Initial reads, additional pages, and HTML/PDF/image preparation share an icon-only loading spinner that exposes its label to assistive technology and respects reduced-motion preferences; every wait before content exists centres the spinner in the pane, so opening a file shows one spinner in one position until the body appears. Loaded pages stay visible while another page loads. The PDF body loads its package-local `client.pdf.js` chunk only when a PDF preview mounts; PDF.js, its Worker source, and embedded support data stay out of the startup `client.js`. PDF pages fill the pane's width edge to edge as one vertical sequence and render lazily near the viewport; an unrendered page holds its place as a quiet 3:4 placeholder block. PDF.js’s official TextLayerBuilder manages selection boundaries and normalized copying over an aligned text layer. Its companion styles keep blank line breaks unhighlighted; alignment accounts for PDF page units and viewport resizing, and page disposal cancels both layers. Image-only PDFs contain no selectable text. Code previews show source line numbers by default without including them in copied text; plain text uses the same font size and line height as code. Code sits on the pane's own background rather than the chat card's fill; its banner is adjacent to a full-height inner scrollport, so both scrollbars begin below the copy control. +Initial reads, additional pages, and HTML/PDF/image preparation share an icon-only loading spinner that exposes its label to assistive technology and respects reduced-motion preferences; every wait before content exists centres the spinner in the pane, so opening a file shows one spinner in one position until the body appears. Loaded pages stay visible while another page loads. The PDF body loads its package-local `client.pdf.js` chunk only when a PDF preview mounts; PDF.js, its Worker source, and embedded support data stay out of the startup `client.js`. PDF pages fill the pane's width edge to edge as one vertical sequence and render lazily near the viewport; an unrendered page holds its place as a quiet 3:4 placeholder block. PDF.js’s official TextLayerBuilder manages selection boundaries and normalized copying over an aligned text layer. Its companion styles keep blank line breaks unhighlighted; alignment accounts for PDF page units, page rotation, and viewport resizing, and page disposal cancels both layers. Image-only PDFs contain no selectable text. Code previews show source line numbers by default without including them in copied text; plain text uses the same font size and line height as code. Code sits on the pane's own background rather than the chat card's fill; its banner is adjacent to a full-height inner scrollport, so both scrollbars begin below the copy control. ## Office preview diff --git a/packages/client/ui-sidebar-documentpreview/README.zh.md b/packages/client/ui-sidebar-documentpreview/README.zh.md index 4d616fe192..94de2f75e5 100644 --- a/packages/client/ui-sidebar-documentpreview/README.zh.md +++ b/packages/client/ui-sidebar-documentpreview/README.zh.md @@ -56,7 +56,7 @@ PNG、JPEG、GIF、WebP、BMP、ICO 和 SVG 通过 Blob URL 在 `` 静态 共享文案来自 `sidebarDocumentPreview`；各内置渲染器拥有自己的本地化标签。PDF 与转换后的 Office 预览在浅色模式下使用石墨灰底色，在深色模式下使用哑黑底色，页面带有轻微阴影并保留文档原色。 -首次读取、追加页及 HTML/PDF/图片准备共用仅图标的加载 spinner，其标签暴露给辅助技术，并遵循减少动态效果偏好；内容出现前的每个等待都把 spinner 居中在面板中，打开文件到正文出现始终是同一位置的一个 spinner。下一页加载期间保留已显示的内容。PDF 正文仅在 PDF 预览挂载时加载包内 `client.pdf.js` chunk；PDF.js、Worker 源码和内嵌支持数据不会进入启动 `client.js`。PDF 页面贴边占满面板宽度，组成一个纵向连续序列并在接近视口时惰性渲染；未渲染的页以安静的 3:4 占位块保持位置。PDF.js 官方 TextLayerBuilder 在与画面重合的文字层上管理选区边界和复制文本规范化。配套样式不高亮空白换行；对齐同时考虑 PDF 页面单位与视口宽度变化，页面释放时取消两层渲染。纯图片 PDF 不包含可选取的文字。代码预览默认显示源码行号，但复制文本不包含行号；纯文本与代码使用相同字号和行高。代码直接坐在分栏自身的背景上，而不是会话卡片的填充色；复制条与占满剩余高度的内部滚动区相邻，因此横纵滚动条都从复制控件下方开始。 +首次读取、追加页及 HTML/PDF/图片准备共用仅图标的加载 spinner，其标签暴露给辅助技术，并遵循减少动态效果偏好；内容出现前的每个等待都把 spinner 居中在面板中，打开文件到正文出现始终是同一位置的一个 spinner。下一页加载期间保留已显示的内容。PDF 正文仅在 PDF 预览挂载时加载包内 `client.pdf.js` chunk；PDF.js、Worker 源码和内嵌支持数据不会进入启动 `client.js`。PDF 页面贴边占满面板宽度，组成一个纵向连续序列并在接近视口时惰性渲染；未渲染的页以安静的 3:4 占位块保持位置。PDF.js 官方 TextLayerBuilder 在与画面重合的文字层上管理选区边界和复制文本规范化。配套样式不高亮空白换行；对齐同时考虑 PDF 页面单位、页面旋转与视口宽度变化，页面释放时取消两层渲染。纯图片 PDF 不包含可选取的文字。代码预览默认显示源码行号，但复制文本不包含行号；纯文本与代码使用相同字号和行高。代码直接坐在分栏自身的背景上，而不是会话卡片的填充色；复制条与占满剩余高度的内部滚动区相邻，因此横纵滚动条都从复制控件下方开始。 ## Office 预览 diff --git a/packages/client/ui-sidebar-documentpreview/src/client/pdf/PdfBody.module.css b/packages/client/ui-sidebar-documentpreview/src/client/pdf/PdfBody.module.css index 29eeb810b2..009b3a359c 100644 --- a/packages/client/ui-sidebar-documentpreview/src/client/pdf/PdfBody.module.css +++ b/packages/client/ui-sidebar-documentpreview/src/client/pdf/PdfBody.module.css @@ -68,6 +68,18 @@ --min-font-size-inv: calc(1 / var(--min-font-size)); } +.text :global(.textLayer)[data-main-rotation="90"] { + transform: rotate(90deg) translateY(-100%); +} + +.text :global(.textLayer)[data-main-rotation="180"] { + transform: rotate(180deg) translate(-100%, -100%); +} + +.text :global(.textLayer)[data-main-rotation="270"] { + transform: rotate(270deg) translateX(-100%); +} + .text :is(span, br) { position: absolute; color: transparent; diff --git a/packages/client/ui-sidebar-documentpreview/src/client/pdf/text.ts b/packages/client/ui-sidebar-documentpreview/src/client/pdf/text.ts index 10321b9c5a..831121ada5 100644 --- a/packages/client/ui-sidebar-documentpreview/src/client/pdf/text.ts +++ b/packages/client/ui-sidebar-documentpreview/src/client/pdf/text.ts @@ -19,7 +19,8 @@ export function pdfTextRenderer(host: HTMLDivElement): RenderPdfText { container.style.setProperty('--scale-round-y', '1px') host.append(container) const resize = (): void => { - container.style.transform = `scale(${host.getBoundingClientRect().width / viewport.width})` + // Width fitting must compose with the viewer's page rotation and translation. + container.style.scale = String(host.getBoundingClientRect().width / viewport.width) } const observer = new ResizeObserver(resize) observer.observe(host) diff --git a/packages/client/ui-sidebar-documentpreview/tests/pdf-fixture.ts b/packages/client/ui-sidebar-documentpreview/tests/pdf-fixture.ts index c01264472a..56fd2ac09c 100644 --- a/packages/client/ui-sidebar-documentpreview/tests/pdf-fixture.ts +++ b/packages/client/ui-sidebar-documentpreview/tests/pdf-fixture.ts @@ -2,17 +2,18 @@ /** * @param userUnit - Page-coordinate unit size as a multiple of 1/72 inch. + * @param rotation - Clockwise page rotation in degrees. * @returns complete PDF bytes; no clocks, external fonts, images, or network references. */ -export function pdfFixture(userUnit = 1): Uint8Array { +export function pdfFixture(userUnit = 1, rotation = 0): Uint8Array { const streams = ['0.9 0.1 0.1 rg 10 10 100 80 re f', '0.1 0.1 0.9 rg 10 10 100 80 re f'] .map(stream => `${stream}\nBT /F1 8 Tf 0 0 0 rg 10 92 Td (Selectable PDF text) Tj ET`) const objects = [ '<< /Type /Catalog /Pages 2 0 R >>', '<< /Type /Pages /Kids [3 0 R 5 0 R] /Count 2 >>', - `<< /Type /Page /Parent 2 0 R /UserUnit ${userUnit} /MediaBox [0 0 120 100] /Resources << /Font << /F1 7 0 R >> >> /Contents 4 0 R >>`, + `<< /Type /Page /Parent 2 0 R /UserUnit ${userUnit} /Rotate ${rotation} /MediaBox [0 0 120 100] /Resources << /Font << /F1 7 0 R >> >> /Contents 4 0 R >>`, `<< /Length ${streams[0]!.length} >>\nstream\n${streams[0]}\nendstream`, - `<< /Type /Page /Parent 2 0 R /UserUnit ${userUnit} /MediaBox [0 0 120 100] /Resources << /Font << /F1 7 0 R >> >> /Contents 6 0 R >>`, + `<< /Type /Page /Parent 2 0 R /UserUnit ${userUnit} /Rotate ${rotation} /MediaBox [0 0 120 100] /Resources << /Font << /F1 7 0 R >> >> /Contents 6 0 R >>`, `<< /Length ${streams[1]!.length} >>\nstream\n${streams[1]}\nendstream`, '<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>', ] diff --git a/packages/client/ui-sidebar-documentpreview/tests/pdf-text.client.spec.ts b/packages/client/ui-sidebar-documentpreview/tests/pdf-text.client.spec.ts index 524d963d88..df52124d7d 100644 --- a/packages/client/ui-sidebar-documentpreview/tests/pdf-text.client.spec.ts +++ b/packages/client/ui-sidebar-documentpreview/tests/pdf-text.client.spec.ts @@ -38,10 +38,10 @@ it('aligns selectable text to the displayed width and removes its observer and D expect(api.create).toHaveBeenCalledWith({ pdfPage: page }) expect(api.render).toHaveBeenCalledWith({ viewport }) expect(observe).toHaveBeenCalledWith(host) - expect((host.firstElementChild as HTMLElement).style.transform).toBe('scale(0.5)') + expect((host.firstElementChild as HTMLElement).style.scale).toBe('0.5') width = 100 resize!([], {} as ResizeObserver) - expect((host.firstElementChild as HTMLElement).style.transform).toBe('scale(0.25)') + expect((host.firstElementChild as HTMLElement).style.scale).toBe('0.25') task.cancel() expect(disconnect).toHaveBeenCalledOnce() expect(api.cancel).toHaveBeenCalledOnce() diff --git a/snapshots/web/document-preview/document.expected.md b/snapshots/web/document-preview/document.expected.md index 870a14a546..28689fa8ca 100644 --- a/snapshots/web/document-preview/document.expected.md +++ b/snapshots/web/document-preview/document.expected.md @@ -39,6 +39,10 @@ - UserUnit 2: selected and copied text aligns with the canvas +## PDF page rotation + +- 90, 180, 270 degrees: selection and copied text align with canvas ink before and after resizing + ## PDF drag selection - Table selection: forward and backward drags exclude later sections