Merge pull request #6513 from deepseek-harness/turtle/plugin-market-skip-private

feat(plugin-market-generator): skip packages and versions marked private

apps/plugin-market-generator/README.i18n.yaml

@@ -9,8 +9,8 @@
   en: ed2a83d06f6789ef
   zh: e5c1e26486dfefb0
 /plugin-market-index-generator/what-gets-listed:
-  en: 7457f51ab0e41cbe
-  zh: 6c690391aaa7cf0a
+  en: 4a01101246724006
+  zh: 8455f6bc4552b787
 /plugin-market-index-generator/caching-and-repeated-runs:
   en: a62c9bf12310c33c
   zh: 60aa01288f46133c

apps/plugin-market-generator/README.md

@@ -9,7 +9,7 @@ A reference generator for [plugin market index](../../.agents/notes/implemented/
 Write a source list, one entry per line. Blank lines and lines starting with `#` are ignored.
 
 ```text
-# npm packages: every non-deprecated bundle version is listed
+# npm packages: every bundle version that is neither deprecated nor private is listed
 @scope/dsh-example
 dsh-another-plugin
 # Git repositories: release tags and the default branch are listed
@@ -43,9 +43,9 @@ The exit status is 0 when every source was read, 1 when a source could not be re
 
 ## What gets listed
 
-Each package becomes one plugin with one or more versions. A version is listed only when its `package.json` declares `dsh.bundle` with `patch` set to a file path or a list of file paths.
+Each package becomes one plugin with one or more versions. A version is listed only when its `package.json` declares `dsh.bundle` with `patch` set to a file path or a list of file paths, and does not set `"private": true`. A Git package whose `package.json` on the default branch sets `"private": true` is not listed, and neither are its tagged versions.
 
-- **npm package**: every version in the registry document that is not deprecated, declares a bundle, and has a publish time. Versions come from the registry document alone; only the tarball of the highest listed version is downloaded, for its display fields. Its `package.json` must name the requested package and version.
+- **npm package**: every version in the registry document that is neither deprecated nor private, declares a bundle, and has a publish time. Versions come from the registry document alone; only the tarball of the highest listed version is downloaded, for its display fields. Its `package.json` must name the requested package and version.
 - **Git repository**: bundle packages are found on the default branch, in this order:
   1. the root `package.json` declares `dsh.bundle`: the root package only;
   2. `package.json` `workspaces` or `pnpm-workspace.yaml` `packages` match packages that declare `dsh.bundle`: those packages. Each glob segment is a literal name, a name with `*`, or `**`; negated globs are ignored, and a glob that matches nothing is reported;

apps/plugin-market-generator/README.zh.md

@@ -9,7 +9,7 @@
 编写来源列表，每行一个条目。生成器忽略空行和以 `#` 开头的行。
 
 ```text
-# npm packages: every non-deprecated bundle version is listed
+# npm packages: every bundle version that is neither deprecated nor private is listed
 @scope/dsh-example
 dsh-another-plugin
 # Git repositories: release tags and the default branch are listed
@@ -43,9 +43,9 @@ pnpm --filter @deepseek-ai/dsh-plugin-market-generator start \
 
 ## 列出的内容
 
-每个包成为一个插件，带有一个或多个版本。只有 `package.json` 声明了 `dsh.bundle`、且其 `patch` 是一个文件路径或文件路径列表的版本才会列出。
+每个包成为一个插件，带有一个或多个版本。只有 `package.json` 声明了 `dsh.bundle`、其 `patch` 是一个文件路径或文件路径列表、且没有设置 `"private": true` 的版本才会列出。默认分支上的 `package.json` 设置了 `"private": true` 的 Git 包不会列出，它在各个标签上的版本也不列出。
 
-- **npm 包**：注册表文档中未弃用、声明了组合包（bundle）且有发布时间的每个版本。版本只取自注册表文档；生成器只下载最高列出版本的 tarball，用于读取展示字段。该 tarball 的 `package.json` 必须与所请求的包名和版本一致。
+- **npm 包**：注册表文档中未弃用、未设为私有、声明了组合包（bundle）且有发布时间的每个版本。版本只取自注册表文档；生成器只下载最高列出版本的 tarball，用于读取展示字段。该 tarball 的 `package.json` 必须与所请求的包名和版本一致。
 - **Git 仓库**：生成器按以下顺序在默认分支上查找组合包：
   1. 根 `package.json` 声明了 `dsh.bundle`：仅使用根包；
   2. `package.json` 的 `workspaces` 或 `pnpm-workspace.yaml` 的 `packages` 匹配到声明了 `dsh.bundle` 的包：使用这些包。每个 glob 段可以是字面名称、含 `*` 的名称或 `**`；生成器忽略否定 glob，并报告未匹配任何内容的 glob；

apps/plugin-market-generator/src/cli.ts

@@ -3,10 +3,10 @@
  *
  * Usage: plugin-market-generator --input <list> --output <index.json> --title <text> [options]
  *
- * The list holds one npm package name or https Git repository URL per line. The generator lists every non-deprecated
- * npm bundle version, and the versions at Git release tags and the default branch, never runs package code, and writes
- * a validated format-1 index. Unchanged releases and commits are read from the cache directory, and the index file is
- * rewritten only when its content changes.
+ * The list holds one npm package name or https Git repository URL per line. The generator lists every npm bundle
+ * version that is neither deprecated nor private, and the versions at Git release tags and the default branch of
+ * packages that are not private, never runs package code, and writes a validated format-1 index. Unchanged releases
+ * and commits are read from the cache directory, and the index file is rewritten only when its content changes.
  */
 import { existsSync, lstatSync, readdirSync, readFileSync, rmSync } from 'node:fs'
 import { dirname, join, resolve } from 'node:path'

apps/plugin-market-generator/src/generate.ts

@@ -6,7 +6,7 @@ import semver from 'semver'
 import { cacheKey, gitRefs, npmPackument, npmTarball, withGitObjects, type FetchLimits, type GitObjects, type GitTransports } from './fetchers.ts'
 import { readJsonFile, writeIfChanged, type PackageFiles } from './files.ts'
 import { GENERATED_ICON, readDisplayMetadata, type Display, type IconFile } from './metadata.ts'
-import { dshRange, findBundlePackages, isBundle, readManifest, type Manifest } from './packages.ts'
+import { dshRange, findBundlePackages, isBundle, isPrivate, readManifest, type Manifest } from './packages.ts'
 import type { Source } from './sources.ts'
 
 /** One generated version, in the field order of the format. */
@@ -59,7 +59,7 @@ interface Release {
 /** Base URL only used to validate relative icon references. */
 const VALIDATION_URL = 'https://index.invalid/index.json'
 /** Part of every cache key; change it when the way entries are derived changes, so cached results are rebuilt. */
-const CACHE_VERSION = 7
+const CACHE_VERSION = 8
 
 /** Why the index format rejects these plugins, or undefined when it accepts them. */
 function invalid(plugins: unknown): string | undefined {
@@ -154,8 +154,8 @@ async function readDisplay(
 }
 
 /**
- * Read every non-deprecated bundle version from the registry document, and the display fields from the tarball of the
- * highest one, through a cache keyed by the listed versions.
+ * Read every bundle version from the registry document that is neither deprecated nor private, and the display fields
+ * from the tarball of the highest one, through a cache keyed by the listed versions.
  */
 async function readNpm(source: Extract<Source, { kind: 'npm' }>, options: GenerateOptions, cache: Cache): Promise<GeneratedEntry[]> {
   const packument = await npmPackument(options.registry, source.name, options.limits)
@@ -164,12 +164,12 @@ async function readNpm(source: Extract<Source, { kind: 'npm' }>, options: Genera
   const releases = Object.entries(packument.versions ?? {}).flatMap(([version, manifest]) => {
     const published = packument.time?.[version]
     const deprecated = typeof manifest.deprecated === 'string' && manifest.deprecated !== ''
-    if (deprecated || !isBundle(manifest) || typeof published !== 'string') return []
+    if (deprecated || isPrivate(manifest) || !isBundle(manifest) || typeof published !== 'string') return []
     return releaseOf(source.name, manifest, version, Math.floor(Date.parse(published) / 1000), npmSource, warn) ?? []
   })
   const highest = entryOf(source.name, releases).versions[0]?.version
   if (highest === undefined) {
-    warn('no published version declares dsh.bundle')
+    warn('no published version that is neither deprecated nor private declares dsh.bundle')
     return []
   }
   const file = cache.file('npm', options.registry, options.inlineIconBytes, source.name, releases.map(release => release.version.version).sort())
@@ -261,7 +261,7 @@ async function readGit(source: Extract<Source, { kind: 'git' }>, options: Genera
       const files = await objects.tree(commit, path)
       const manifest = await readManifest(files, '')
       const gitSource = { type: 'git', url: source.url, commit, ...path === '' ? {} : { path } }
-      release = isBundle(manifest) && typeof manifest?.name === 'string' && typeof manifest.version === 'string'
+      release = isBundle(manifest) && !isPrivate(manifest) && typeof manifest?.name === 'string' && typeof manifest.version === 'string'
         ? releaseOf(manifest.name, manifest, manifest.version, await objects.committed(commit), gitSource, warn) ?? null : null
     } catch (error) {
       warn(`${commit}:${path} skipped: ${(error as Error).message}`)
@@ -284,16 +284,19 @@ async function readGit(source: Extract<Source, { kind: 'git' }>, options: Genera
   }
 }
 
-/** Names of the bundle packages on the default branch, by package directory. */
+/** Names of the bundle packages on the default branch that are not private, by package directory. */
 async function headPackages(files: PackageFiles, warn: (message: string) => void): Promise<{ path: string; name: string }[]> {
   const paths = await findBundlePackages(files, warn)
   if (paths.length === 0) warn('no package declares dsh.bundle')
   const packages: { path: string; name: string }[] = []
+  let privates = 0
   for (const path of paths) {
-    const name = (await readManifest(files.within(path), ''))?.name
-    if (typeof name === 'string') packages.push({ path, name })
+    const manifest = await readManifest(files.within(path), '')
+    if (isPrivate(manifest)) privates++
+    else if (typeof manifest?.name === 'string') packages.push({ path, name: manifest.name })
     else warn(`${posix.join(path, 'package.json')} has no name`)
   }
+  if (privates > 0 && privates === paths.length) warn('every package that declares dsh.bundle is private')
   return packages
 }
 

apps/plugin-market-generator/src/packages.ts

@@ -15,6 +15,7 @@ export interface Manifest {
   readonly repository?: unknown
   readonly keywords?: unknown
   readonly deprecated?: unknown
+  readonly private?: unknown
   readonly workspaces?: unknown
   readonly exports?: unknown
   readonly engines?: unknown
@@ -63,6 +64,15 @@ export function isBundle(manifest: Manifest | undefined): boolean {
   return typeof bundle.patch === 'string' || (Array.isArray(bundle.patch) && bundle.patch.every(file => typeof file === 'string'))
 }
 
+/**
+ * Whether a manifest sets `"private": true`, which keeps its package or version out of an index.
+ * @param manifest - parsed manifest, or undefined for a directory without one.
+ * @returns true only when `private` is the boolean `true`.
+ */
+export function isPrivate(manifest: Manifest | undefined): boolean {
+  return manifest?.private === true
+}
+
 /** Whether installation reads `range` as a SemVer range; an empty range and other `workspace:` ranges never match. */
 const valid = (range: string): boolean => range !== '' && !range.startsWith('workspace:') && semver.validRange(range) !== null
 const invalidRange = (range: string): Error => new Error(`DSH range ${JSON.stringify(range)} is not a valid SemVer range`)

apps/plugin-market-generator/tests/discovery.spec.ts

@@ -1,7 +1,7 @@
 import { describe, expect, it } from 'vitest'
 import { PackageFiles } from '../src/files.ts'
 import { readDisplayMetadata } from '../src/metadata.ts'
-import { dshRange, findBundlePackages, isBundle, readManifest, type Manifest } from '../src/packages.ts'
+import { dshRange, findBundlePackages, isBundle, isPrivate, readManifest, type Manifest } from '../src/packages.ts'
 import { parseSources } from '../src/sources.ts'
 
 /** An in-memory tree from relative path → file contents. */
@@ -54,6 +54,11 @@ describe('isBundle and dshRange', () => {
     expect(isBundle(undefined)).toBe(false)
   })
 
+  it('reads only the boolean true as private', () => {
+    expect([true, 'true', false, 1, undefined].map(value => isPrivate({ private: value }))).toEqual([true, false, false, false, false])
+    expect(isPrivate(undefined)).toBe(false)
+  })
+
   it('copies engines.dsh, else joins DSH peer ranges, and refuses ranges installation rejects', () => {
     expect(dshRange({ engines: { dsh: '>=0.2.0' }, peerDependencies: { '@deepseek-ai/dsh': '1' } })).toBe('>=0.2.0')
     expect(dshRange({ peerDependencies: { react: '18', '@deepseek-ai/dsh-agent': 'workspace:*', '@deepseek-ai/dsh': '*' } })).toBeUndefined()

apps/plugin-market-generator/tests/generate.spec.ts

@@ -87,12 +87,13 @@ function options(registryUrl: string, cacheDir: string, lines: string[], warning
   }
 }
 
-it('lists every non-deprecated bundle version from the registry document and reads display fields from the highest tarball', async () => {
+it('lists every bundle version from the registry document that is neither deprecated nor private and reads display fields from the highest tarball', async () => {
   const packages: Packages = {
     'dsh-a': {
       '0.9.0': { time: '2026-01-01T00:00:00.000Z' },
       '1.0.0': { time: '2026-02-01T00:00:00.000Z', manifest: { ...BUNDLE, keywords: ['old'], engines: { dsh: '>=0.2.0' }, peerDependencies: { '@deepseek-ai/dsh': '9' } } },
       '1.1.0': { time: '2026-03-01T00:00:00.000Z', manifest: { ...BUNDLE, deprecated: 'broken' } },
+      '1.1.5': { time: '2026-03-15T00:00:00.000Z', manifest: { ...BUNDLE, private: true } },
       '1.2.0': {
         time: '2026-04-01T00:00:00.000Z',
         manifest: {
@@ -117,6 +118,7 @@ it('lists every non-deprecated bundle version from the registry document and rea
     },
     'dsh-b': { '1.0.0': { time: '2026-06-02T00:00:00.000Z', manifest: { ...BUNDLE, author: 'Bob' }, tarball: await pack({ 'package.json': bundleManifest('dsh-b', '1.0.0') }) } },
     'plain-lib': { '1.0.0': { time: '2026-06-01T00:00:00.000Z' } },
+    'dsh-private': { '1.0.0': { time: '2026-06-01T00:00:00.000Z', manifest: { ...BUNDLE, private: true } } },
     'dsh-imposter': { '1.0.0': { time: '2026-06-01T00:00:00.000Z', manifest: BUNDLE, tarball: await pack({ 'package.json': bundleManifest('dsh-other', '1.0.0') }) } },
     'dsh-tampered': { '1.0.0': { time: '2026-06-01T00:00:00.000Z', manifest: BUNDLE, tarball: await pack({ 'package.json': bundleManifest('dsh-tampered', '1.0.0') }), integrity: 'sha512-AAAA' } },
     'dsh-unsigned': { '1.0.0': { time: '2026-06-01T00:00:00.000Z', manifest: BUNDLE } },
@@ -129,7 +131,7 @@ it('lists every non-deprecated bundle version from the registry document and rea
   const server = await registry(packages)
   const cacheDir = scratch()
   const warnings: string[] = []
-  const lines = ['dsh-a', 'dsh-b', 'plain-lib', 'dsh-imposter', 'dsh-tampered', 'dsh-unsigned', 'dsh-huge', 'dsh-bomb', 'dsh-no-tarball', 'dsh-unpublished', 'dsh-missing']
+  const lines = ['dsh-a', 'dsh-b', 'plain-lib', 'dsh-private', 'dsh-imposter', 'dsh-tampered', 'dsh-unsigned', 'dsh-huge', 'dsh-bomb', 'dsh-no-tarball', 'dsh-unpublished', 'dsh-missing']
   const first = await generate(options(server.url, cacheDir, lines, warnings))
   const npm = { type: 'npm', registry: server.url }
   expect(first.index).toMatchObject({ format: 1, title: 'Test Market', description: 'For tests' })
@@ -160,8 +162,9 @@ it('lists every non-deprecated bundle version from the registry document and rea
   expect(warnings).toEqual(expect.arrayContaining([
     'dsh-a: [email protected] skipped: DSH range "workspace:>=1" is not a valid SemVer range',
     '[email protected]: locale file locale/x_y.json ignored: its name is not a language id',
-    'plain-lib: no published version declares dsh.bundle',
-    'dsh-unpublished: no published version declares dsh.bundle',
+    'plain-lib: no published version that is neither deprecated nor private declares dsh.bundle',
+    'dsh-private: no published version that is neither deprecated nor private declares dsh.bundle',
+    'dsh-unpublished: no published version that is neither deprecated nor private declares dsh.bundle',
   ]))
 
   // Unchanged version lists come from the cache; a failing source keeps its last result for the same registry only.

apps/plugin-market-generator/tests/git.spec.ts

@@ -76,6 +76,7 @@ it('lists Git versions from release tags and the default branch by reading objec
     '.gitattributes': '* filter=marker\n',
     'packages/a/package.json': bundle('@s/a', '1.0.0'),
     'packages/b/package.json': bundle('@s/b', '0.9.0'),
+    'packages/e/package.json': bundle('@s/e', '1.0.0'),
   })
   const first = repo.commit(1790000000)
   repo.git(['tag', 'v1.0.0'])
@@ -96,6 +97,8 @@ it('lists Git versions from release tags and the default branch by reading objec
     'packages/a/icon.svg': '<svg/>',
     'packages/b/package.json': bundle('@s/b', '2.0.0', { keywords: ['head'] }),
     'packages/d/package.json': JSON.stringify({ ...JSON.parse(bundle('@s/d', '0.1.0')), peerDependencies: { '@deepseek-ai/dsh': 'workspace:>=1' } }),
+    // A package that turns private on the default branch is not listed, not even at its earlier public tag.
+    'packages/e/package.json': bundle('@s/e', '1.0.1', { private: true }),
   })
   const head = repo.commit(1790000300)
   repo.git(['tag', 'v2.0.0'])
@@ -131,7 +134,7 @@ it('lists Git versions from release tags and the default branch by reading objec
   expect(updated.index.plugins[1]).not.toHaveProperty('metadata')
 })
 
-it('reads a root package, reports repositories without one, and refuses transports that are not allowed', GIT_TEST, async () => {
+it('reads a root package, reports repositories without a public one, and refuses transports that are not allowed', GIT_TEST, async () => {
   const repo = repository()
   gitHome(repo.dir)
   const warnings: string[] = []
@@ -149,12 +152,20 @@ it('reads a root package, reports repositories without one, and refuses transpor
   repo.write({ 'package.json': JSON.stringify({ version: '1.0.0', dsh: { bundle: { patch: 'p.yml' } } }) })
   repo.commit(1790000100)
   expect(await run()).toEqual([[], []])
+  // A private package is not listed, and a release tag on a private manifest lists no version.
+  repo.write({ 'package.json': bundle('dsh-root', '0.9.0', { private: true }) })
+  repo.commit(1790000150)
+  repo.git(['tag', 'v0.9.0'])
+  expect(await run()).toEqual([[], []])
   repo.write({ 'package.json': bundle('dsh-root', '1.0.0') })
   const commit = repo.commit(1790000200)
   expect(await run()).toEqual([[{
     name: 'dsh-root', title: 'dsh-root', versions: [{ version: '1.0.0', publishTimestamp: 1790000200, source: { type: 'git', url: REPOSITORY, commit } }],
   }], []])
-  expect(warnings).toEqual([`${REPOSITORY}: ${REPOSITORY}: no HEAD commit`, expect.stringContaining('not allowed') as string, `${REPOSITORY}: no package declares dsh.bundle`, `${REPOSITORY}: package.json has no name`])
+  expect(warnings).toEqual([
+    `${REPOSITORY}: ${REPOSITORY}: no HEAD commit`, expect.stringContaining('not allowed') as string, `${REPOSITORY}: no package declares dsh.bundle`,
+    `${REPOSITORY}: package.json has no name`, `${REPOSITORY}: every package that declares dsh.bundle is private`,
+  ])
 })
 
 it('skips a release tag it cannot read, refuses oversized manifests, and ignores Git overrides from the environment', GIT_TEST, async () => {

docs/user/guide/plugin-market.i18n.yaml

@@ -21,8 +21,8 @@
   en: b85b82b22c1e14b8
   zh: 920c0d6f20956c07
 /plugin-markets/generate-an-index-with-the-reference-generator:
-  en: 57a48912c00e1c85
-  zh: 69f2934eed0fbadf
+  en: 6501406f530ffe33
+  zh: 700acae3f466c985
 /plugin-markets/index-format-reference:
   en: d058d12c27433f64
   zh: f43b85e4934817f3

docs/user/guide/plugin-market.md

@@ -97,7 +97,7 @@ Run it from a DSH checkout:
 pnpm --filter @deepseek-ai/dsh-plugin-market-generator start --input plugins.txt --output site/index.json --title "Team plugins"
 ```
 
-For an npm package, the index lists every version that is not deprecated and declares `dsh.bundle`. For a Git repository, it finds the bundle packages on the default branch and lists their versions at `vX.Y.Z`, `X.Y.Z`, and `<name>@X.Y.Z` tags, plus each package's version on the default branch. Titles, descriptions, icons, authors, and homepages come from each plugin's highest version. Paths are relative to the directory where you run pnpm.
+For an npm package, the index lists every version that is neither deprecated nor private and declares `dsh.bundle`. For a Git repository, it finds the bundle packages on the default branch whose `package.json` does not set `"private": true` and lists their versions at `vX.Y.Z`, `X.Y.Z`, and `<name>@X.Y.Z` tags, plus each package's version on the default branch. Titles, descriptions, icons, authors, and homepages come from each plugin's highest version. Paths are relative to the directory where you run pnpm.
 
 The generator only downloads and reads files; it never installs packages or runs their code. It keeps a cache in `.plugin-market-cache` next to the list, so a repeated run downloads only new versions and commits. Publish the output directory, `index.json` together with its `icons/` directory, as described above.
 

docs/user/guide/plugin-market.zh.md

@@ -97,7 +97,7 @@ https://gitlab.example.com/team/dsh-plugins
 pnpm --filter @deepseek-ai/dsh-plugin-market-generator start --input plugins.txt --output site/index.json --title "Team plugins"
 ```
 
-对于 npm 包，索引列出所有未弃用且声明了 `dsh.bundle` 的版本。对于 Git 仓库，生成器在默认分支上查找组合包，列出它们在 `vX.Y.Z`、`X.Y.Z` 和 `<name>@X.Y.Z` 标签处的版本，以及每个包在默认分支上的版本。标题、描述、图标、作者和主页取自每个插件的最高版本。路径相对于你运行 pnpm 的目录。
+对于 npm 包，索引列出所有未弃用、未设为私有且声明了 `dsh.bundle` 的版本。对于 Git 仓库，生成器在默认分支上查找 `package.json` 没有设置 `"private": true` 的组合包，列出它们在 `vX.Y.Z`、`X.Y.Z` 和 `<name>@X.Y.Z` 标签处的版本，以及每个包在默认分支上的版本。标题、描述、图标、作者和主页取自每个插件的最高版本。路径相对于你运行 pnpm 的目录。
 
 生成器只下载和读取文件，从不安装包或运行其代码。它在列表旁的 `.plugin-market-cache` 中保存缓存，因此再次运行时只下载新的版本和提交。按上文所述发布输出目录，即 `index.json` 及其 `icons/` 目录。