diff --git a/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.i18n.yaml index 999a35f13c..980da73e47 100644 --- a/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.md -2026-08-13-credential-records-and-authorization-flows.md: 19de5a5212e12a678a2162275f26130cf54f4f46 -2026-08-13-credential-records-and-authorization-flows.zh.md: ee79248a8c6ae1d652bb280bb9863c01655715c7 +2026-08-13-credential-records-and-authorization-flows.md: a52d85854c8c660b4965d07a13a2dbfefbac5723 +2026-08-13-credential-records-and-authorization-flows.zh.md: 1b6960cd9cff5e7320816a1528e3c621fea41691 diff --git a/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.md b/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.md index 19de5a5212..a52d85854c 100644 --- a/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.md +++ b/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.md @@ -51,7 +51,7 @@ Withdrawal settles an attempt whether or not its flow reacts to the signal. A fl ## Consequences -`.credentials.yaml` gains a version and two sections, and the pre-release flat layout is refused by name with the migration stated in the message — pre-release stance, no compatibility shim. Every fixture in the repo that wrote the flat document was rewritten; the llm suites' fixtures were missed by the record change itself and fixed here. +`.credentials.yaml` gains a version and two sections. A boot upgrades the recognized pre-release flat layout in place — an all-string flat mapping nests verbatim under `refs:` under the writer lock — because a key stored through the Models page by an earlier internal build must survive the layout change without a hand edit and without its model requests failing. Any flat shape the recognizer cannot prove it understands keeps the by-name refusal with the hand migration stated in the message; the parser itself still reads exactly one layout, and the migration step retires with the pre-release stance at the first tagged release. Every fixture in the repo that wrote the flat document was rewritten; the llm suites' fixtures were missed by the record change itself and fixed here. `openai-codex` returns to the provider picker and to the Models page directory. Signing in is offered for every installed provider that ships a login, which today is all 38 — 31 collect a key through pi-ai's own prompt, six offer that beside a subscription login, and Codex offers only the subscription login. diff --git a/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.zh.md b/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.zh.md index ee79248a8c..1b6960cd9c 100644 --- a/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-13-credential-records-and-authorization-flows.zh.md @@ -51,7 +51,7 @@ seam 的边缘与写入路径同一纪律。prompt 被拒是结果而非故障 ## Consequences -`.credentials.yaml` 增加了版本与两个分区，发布前的扁平布局被指名拒绝，迁移办法写在报错信息里——发布前立场，不留兼容垫片。仓库中所有写扁平文档的 fixture 都已改写；llm 各套件的 fixture 被记录改动本身漏掉了，在此补上。 +`.credentials.yaml` 增加了版本与两个分区。启动时会把能精确识别的发布前扁平布局原地升级——全字符串的扁平 mapping 在写锁下逐字下沉到 `refs:` 之下——因为早期内测构建经模型页面存下的密钥必须在布局变更后继续可用，不能要求手工编辑，也不能让模型请求失败。识别器无法证明自己理解的扁平形态仍被指名拒绝，迁移办法写在报错信息里；解析器本身始终只读一种布局，迁移步骤将随发布前立场在首个正式版本时移除。仓库中所有写扁平文档的 fixture 都已改写；llm 各套件的 fixture 被记录改动本身漏掉了，在此补上。 `openai-codex` 回到提供方选择器与 Models 页目录。凡是自带登录的已安装提供方都会得到登录入口，而今天这是全部 38 个——31 个经 pi-ai 自己的提示收取密钥，6 个在此之外还提供订阅登录，Codex 只提供订阅登录。 diff --git a/packages/credentials/credentials-local/README.i18n.yaml b/packages/credentials/credentials-local/README.i18n.yaml index b54623d908..df468deda8 100644 --- a/packages/credentials/credentials-local/README.i18n.yaml +++ b/packages/credentials/credentials-local/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/credentials/credentials-local/README.md -README.md: 1b8e4e5d89dc0eb5da2b04b6281c70316262c03b -README.zh.md: 7ba7726e46a6cd5c6b26a2e55e001a750a0e6dd1 +README.md: 7acd89ea24be4da3d2cecc6efe619e73e05bb823 +README.zh.md: 4b62904af23dd7e907f045e0713abba59c2abba9 diff --git a/packages/credentials/credentials-local/README.md b/packages/credentials/credentials-local/README.md index 1b8e4e5d89..7acd89ea24 100644 --- a/packages/credentials/credentials-local/README.md +++ b/packages/credentials/credentials-local/README.md @@ -57,7 +57,7 @@ The document holds credentials only, so every deviation is a rejection rather th A `grant` payload must survive a JSON round trip, enforced in both directions. YAML spells values JSON has none for — `.inf`, alias cycles — and an owner may hand over a `Date` or a `bigint`; either way the store refuses rather than saving something it could not read back exactly as written. -The pre-release layout was a flat mapping with no `version`. It is refused by name, with the entry count and the one edit needed (`version: 1`, nest under `refs:`) — never read as an empty store, which would surface as an authentication failure on the first request instead of at load. +The pre-release layout was a flat mapping with no `version`. A boot that recognizes it exactly — addressable names over non-empty string scalars, no directives — upgrades the document in place under the writer lock: the original lines nest verbatim under `refs:`, so values, comments, and spellings survive byte for byte. Any other flat shape is refused by name, with the entry count and the one edit needed (`version: 1`, nest under `refs:`) — never read as an empty store, which would surface as an authentication failure on the first request instead of at load. A live reload never migrates: a flat document restored mid-run keeps the last good snapshot until the next boot. Writes patch the parsed document rather than rebuilding it, so comments and the formatting of every untouched entry survive. A comment directly above an entry is that entry's annotation and is removed with it. Every write first re-reads the document under the cross-process writer lock of [`dsh-atomic-write`](../../util/atomic-write/README.md) and publishes anything it had not observed, then commits atomically with mode `0600` under an owner-only (`0700`) directory — so a concurrent writer or an external edit inside the watcher's debounce window is folded in rather than overwritten. An on-disk document that no longer parses fails the write instead of overwriting content the provider could not understand. diff --git a/packages/credentials/credentials-local/README.zh.md b/packages/credentials/credentials-local/README.zh.md index 7ba7726e46..4b62904af2 100644 --- a/packages/credentials/credentials-local/README.zh.md +++ b/packages/credentials/credentials-local/README.zh.md @@ -57,7 +57,7 @@ records: `grant` 的 payload 必须能经受 JSON 往返，读写两个方向都强制。YAML 能拼写出 JSON 没有的值——`.inf`、别名环——拥有者也可能递来 `Date` 或 `bigint`；无论哪种，存储都选择拒绝，而不是存下一个自己无法逐字读回的东西。 -发布前的旧布局是没有 `version` 的扁平 mapping。它会被指名拒绝，并给出条目数与唯一需要做的编辑（`version: 1`，条目下沉到 `refs:`）——绝不当作空存储读过去，否则它会以第一次请求认证失败的形式出现，而不是在加载时。 +发布前的旧布局是没有 `version` 的扁平 mapping。启动时若能精确识别它——可寻址名称对非空字符串标量、且没有文档指令——就在写锁下原地升级：原有各行逐字下沉到 `refs:` 之下，值、注释与拼写逐字节保留。其余任何扁平形态都会被指名拒绝，并给出条目数与唯一需要做的编辑（`version: 1`，条目下沉到 `refs:`）——绝不当作空存储读过去，否则它会以第一次请求认证失败的形式出现，而不是在加载时。热重载从不迁移：运行中被恢复出来的扁平文档只会保住上一份完好快照，直到下次启动。 写入是对已解析文档打补丁而不是重建，因此注释与所有未触及条目的排版都会保留。直接位于某条目上方的注释属于该条目的注解，会随它一起删除。每次写入都先在 [`dsh-atomic-write`](../../util/atomic-write/README.md) 的跨进程写锁下重读文档、把此前未观察到的一切发布出去，再在仅属主可访问（`0700`）的目录下以 `0600` 权限原子提交——因此并发写入者、或落在 watcher 防抖窗口内的外部编辑会被并入，而不是被覆盖。磁盘上已经无法解析的文档会让写入失败，而不是覆盖提供方读不懂的内容。 diff --git a/packages/credentials/credentials-local/src/index.ts b/packages/credentials/credentials-local/src/index.ts index 20a8313344..9bdece7bc6 100644 --- a/packages/credentials/credentials-local/src/index.ts +++ b/packages/credentials/credentials-local/src/index.ts @@ -226,6 +226,45 @@ export function parseCredentialsDocument(text: string, filename: string): Creden return { refs: parseRefs(fields['refs'], filename), records: parseRecords(fields['records'], filename) } } +/** + * Render the version-1 layout for a pre-release flat document, or `undefined` + * for anything else. The flat layout is recognized exactly — a non-empty + * top-level mapping of addressable reference names to non-empty string + * scalars, with no `version` key and no document directives — and the rewrite + * nests the original lines verbatim under `refs:` at two spaces' indent, so + * comments, blank lines, and each value's spelling survive byte for byte. + * Anything the recognizer declines keeps {@link parseCredentialsDocument}'s + * loud rejection: a document this build cannot prove it understands is never + * rewritten. Remove with the pre-release stance at the first tagged release. + * @param text - the document's text. + * @returns the migrated text, or `undefined` when the text is not the recognized flat layout. + */ +export function renderFlatLayoutMigration(text: string): string | undefined { + const document = parseDocument(text, { prettyErrors: true, uniqueKeys: true }) + if (document.errors.length > 0) return undefined + const flat = document.contents + if (!isMap(flat) || flat.items.length === 0) return undefined + for (const line of text.split('\n')) { + // A directive or document marker would not survive being indented into + // the `refs:` block; no shipped writer ever emitted one here. + if (/^(%|---|\.\.\.)/.test(line)) return undefined + } + for (const pair of flat.items) { + if (!isScalar(pair.key) || typeof pair.key.value !== 'string' || pair.key.value === 'version') return undefined + try { + credentialRef(pair.key.value) + } catch { + // Only credentialRef's rejection of a non-POSIX name lands here; the + // flat reader refused such a key too, so this is not the recognized + // layout and the loud rejection stands. + return undefined + } + if (!isScalar(pair.value) || typeof pair.value.value !== 'string' || pair.value.value.length === 0) return undefined + } + const body = text.split('\n').map(line => (line.length === 0 ? line : ` ${line}`)).join('\n') + return `version: ${DOCUMENT_VERSION}\nrefs:\n${body}${text.endsWith('\n') ? '' : '\n'}` +} + /** Admit a `refs` section: POSIX-identifier keys over non-empty string values. */ function parseRefs(section: unknown, filename: string): Map { const entries = new Map() @@ -764,7 +803,10 @@ export class LocalCredentialProvider extends CredentialProvider { /** * Boot read: an absent file is an empty store; an invalid one fails the * plugin's activation, because a credentials document that exists but - * cannot be trusted must never be treated as "no credentials stored". + * cannot be trusted must never be treated as "no credentials stored". The + * one exception is the recognized pre-release flat layout, which is + * upgraded in place first — a key stored by an earlier build must survive + * the layout change without a hand edit. */ private async loadInitial(): Promise { await assertOwnerOnly(this.spec.filename) @@ -775,12 +817,44 @@ export class LocalCredentialProvider extends CredentialProvider { if (!isENOENT(error)) throw error return } + if (renderFlatLayoutMigration(text) !== undefined) text = await this.migrateFlatDocument() const document = parseCredentialsDocument(text, this.spec.filename) this.values = document.refs this.records = document.records this.text = text } + /** + * One-shot upgrade of the recognized pre-release flat layout, before the + * watcher exists. The rewrite runs under the document's writer lock and + * re-reads first — a concurrent boot may have migrated already — and + * whatever the re-read finds that is not the flat layout is returned + * untouched for the ordinary parse. Values are carried verbatim; only the + * enclosing layout changes. Remove with the pre-release stance at the + * first tagged release. + * @returns the document text this boot should parse. + */ + private async migrateFlatDocument(): Promise { + return withFileLock(this.spec.filename, async () => { + const current = await readFile(this.spec.filename, 'utf8') + const migrated = renderFlatLayoutMigration(current) + /* v8 ignore next 2 -- the losing side of the cross-process migration race: + another boot rewrote the document between the unlocked recognize and + this lock. That interleaving cannot be scheduled deterministically + through a whole boot (migration.spec drives it best-effort); the + decision itself is the recognizer's covered versioned-document decline. */ + if (migrated === undefined) return current + // 0600: a document holding secrets is never world-readable. + await writeFileAtomic(this.spec.filename, migrated, { mode: 0o600, dirMode: 0o700 }) + this.ctx.logger.info( + 'credentials-local: migrated %s to the version %d layout; values are unchanged', + this.spec.filename, + DOCUMENT_VERSION, + ) + return migrated + }, { waitMs: DOCUMENT_LOCK_WAIT_MS }) + } + /* jscpd:ignore-start -- same deliberate mirror of settings-file's reload and reconcile policy: warn-and-keep on a reload, throw on a write, invariant failures propagate. */ diff --git a/packages/credentials/credentials-local/tests/local.spec.ts b/packages/credentials/credentials-local/tests/local.spec.ts index 2158790b2e..230015fe9d 100644 --- a/packages/credentials/credentials-local/tests/local.spec.ts +++ b/packages/credentials/credentials-local/tests/local.spec.ts @@ -230,12 +230,19 @@ describe('document validation', () => { it.each([ ['a non-mapping root', 'just a string\n', /must be a mapping/], ['a sequence root', '- DSH_CRED_TEST\n', /must be a mapping/], - // The pre-release layout is refused by name rather than read as an empty - // store: a silently ignored document would surface as an authentication - // failure on the first request instead of at load. - ['the pre-release flat layout', 'DSH_CRED_TEST: stored\n', /nest the existing 1 entry under/], - ['the pre-release flat layout with several entries', 'DSH_CRED_TEST: a\nDSH_CRED_OTHER: b\n', + // A flat document the boot migration cannot prove it understands is + // refused by name rather than read as an empty store or rewritten: a + // silently ignored document would surface as an authentication failure on + // the first request instead of at load. (The recognized all-string flat + // layout upgrades in place instead — migration.spec owns that path.) + ['the flat layout with a non-string value', 'DSH_CRED_TEST: [nope]\n', /nest the existing 1 entry under/], + ['the flat layout with several entries and a non-string value', 'DSH_CRED_TEST: a\nDSH_CRED_OTHER: [b]\n', /nest the existing 2 entries under/], + ['the flat layout with an empty value', 'DSH_CRED_TEST: ""\n', /pre-release flat layout/], + ['the flat layout with an unaddressable key', 'not-a-ref: value\n', /pre-release flat layout/], + ['the flat layout with a non-string key', '1: a\n', /pre-release flat layout/], + ['the flat layout under document directives', '%YAML 1.2\n---\nDSH_CRED_TEST: a\n', + /pre-release flat layout/], ['a future version', 'version: 2\nrefs: {}\n', /this build reads version 1/], ['an unknown top-level key', 'version: 1\nsecrets: {}\n', /unknown top-level key "secrets"/], ['a non-mapping refs section', 'version: 1\nrefs: nope\n', /"refs" .* must be a mapping/], diff --git a/packages/credentials/credentials-local/tests/migration.spec.ts b/packages/credentials/credentials-local/tests/migration.spec.ts new file mode 100644 index 0000000000..30fec951e3 --- /dev/null +++ b/packages/credentials/credentials-local/tests/migration.spec.ts @@ -0,0 +1,140 @@ +// One-shot boot upgrade of the pre-release flat layout: a key stored by an +// earlier build must survive the versioned-document change without a hand +// edit, byte for byte, while everything the recognizer cannot prove flat +// keeps the loud rejection local.spec exercises. +import { afterEach, describe, expect, it } from 'vitest' +import { Context } from '@deepseek-ai/cordis' +import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { credentialRef } from '@deepseek-ai/dsh-credentials' +import { withFileLock } from '@deepseek-ai/dsh-atomic-write' +import { LocalCredentialProvider, renderFlatLayoutMigration } from '../src/index.ts' + +/** Credential documents are seeded owner-only, exactly as the provider creates them. */ +function writeCredentials(file: string, text: string): Promise { + return writeFile(file, text, { mode: 0o600 }) +} + +const cleanups: Array<() => Promise> = [] + +afterEach(async () => { + while (cleanups.length > 0) await cleanups.pop()!() +}) + +async function tempDir(): Promise { + const dir = await mkdtemp(join(tmpdir(), 'dsh-cred-migration-')) + cleanups.push(() => rm(dir, { recursive: true, force: true })) + return dir +} + +async function boot(config: ConstructorParameters[1]): Promise { + const ctx = new Context() + const fiber = ctx.plugin(LocalCredentialProvider, config) + cleanups.push(async () => { await fiber.dispose() }) + await fiber + return ctx +} + +// Every spelling an earlier build accepted: plain, quoted, block-scalar, a +// comment header, an interior blank line, and a key that happens to spell a +// section name of the versioned layout. +const FLAT = [ + '# keys stored before the versioned layout', + 'DSH_CRED_TEST: stored', + '', + '# annotates the quoted entry', + "DSH_CRED_OTHER: 'quoted value'", + 'DSH_CRED_BLOCK: |', + ' first line', + ' second line', + 'records: tricky', +].join('\n') + '\n' + +const MIGRATED = [ + 'version: 1', + 'refs:', + ' # keys stored before the versioned layout', + ' DSH_CRED_TEST: stored', + '', + ' # annotates the quoted entry', + " DSH_CRED_OTHER: 'quoted value'", + ' DSH_CRED_BLOCK: |', + ' first line', + ' second line', + ' records: tricky', +].join('\n') + '\n' + +describe('flat-layout boot migration', () => { + it('upgrades the flat document in place, byte for byte, and serves its keys', async () => { + const dir = await tempDir() + const path = join(dir, '.credentials.yaml') + await writeCredentials(path, FLAT) + const ctx = await boot({ path, watch: false }) + expect(await readFile(path, 'utf8')).toBe(MIGRATED) + if (process.platform !== 'win32') expect((await stat(path)).mode & 0o777).toBe(0o600) + expect(await ctx.credentials.resolve(credentialRef('DSH_CRED_TEST'))).toEqual({ value: 'stored', source: 'file' }) + expect(await ctx.credentials.resolve(credentialRef('DSH_CRED_OTHER'))) + .toEqual({ value: 'quoted value', source: 'file' }) + expect(await ctx.credentials.resolve(credentialRef('DSH_CRED_BLOCK'))) + .toEqual({ value: 'first line\nsecond line\n', source: 'file' }) + expect(await ctx.credentials.resolve(credentialRef('records'))).toEqual({ value: 'tricky', source: 'file' }) + }) + + it('a second boot reads the migrated document without touching it', async () => { + const dir = await tempDir() + const path = join(dir, '.credentials.yaml') + await writeCredentials(path, FLAT) + await boot({ path, watch: false }) + const ctx = await boot({ path, watch: false }) + expect(await readFile(path, 'utf8')).toBe(MIGRATED) + expect(await ctx.credentials.resolve(credentialRef('DSH_CRED_TEST'))).toEqual({ value: 'stored', source: 'file' }) + }) + + it('yields to a concurrent migrator under the writer lock', async () => { + const dir = await tempDir() + const path = join(dir, '.credentials.yaml') + await writeCredentials(path, FLAT) + const winner = 'version: 1\nrefs:\n DSH_CRED_TEST: winner\n' + let release!: () => void + const held = new Promise((resolve) => { release = resolve }) + let acquired!: () => void + const holding = new Promise((resolve) => { acquired = resolve }) + const holder = withFileLock(path, async () => { + acquired() + await held + }) + await holding + // The boot sees the flat text, then waits for the lock; the "other + // process" completes the migration in the meantime. + const booting = boot({ path, watch: false }) + await writeCredentials(path, winner) + release() + await holder + const ctx = await booting + expect(await readFile(path, 'utf8')).toBe(winner) + expect(await ctx.credentials.resolve(credentialRef('DSH_CRED_TEST'))).toEqual({ value: 'winner', source: 'file' }) + }) + + it('leaves an empty flow mapping alone', async () => { + const dir = await tempDir() + const path = join(dir, '.credentials.yaml') + await writeCredentials(path, '{}\n') + const ctx = await boot({ path, watch: false }) + expect(await readFile(path, 'utf8')).toBe('{}\n') + expect(await ctx.credentials.resolve(credentialRef('DSH_CRED_TEST'))).toBeUndefined() + }) + + it('leaves a comment-only document alone', async () => { + const dir = await tempDir() + const path = join(dir, '.credentials.yaml') + await writeCredentials(path, '# nothing stored yet\n') + const ctx = await boot({ path, watch: false }) + expect(await readFile(path, 'utf8')).toBe('# nothing stored yet\n') + expect(await ctx.credentials.resolve(credentialRef('DSH_CRED_TEST'))).toBeUndefined() + }) + + it('renders a final newline for a document that lacks one', () => { + expect(renderFlatLayoutMigration('DSH_CRED_TEST: bare')).toBe('version: 1\nrefs:\n DSH_CRED_TEST: bare\n') + }) +})