Files
deepseek-harness/.gitea/workflows/sync-upstream.yml
T
mward4 128ca9d8b3
Sync upstream and build images / sync-and-build (push) Successful in 16m13s
ci: only a never-published version can fail the sync run
Runs have been red since 2026-09-19 even though every release shipped: the
plan step re-plans the oldest 11 versions as "stale" on every run (pnpm and
the node base image moved on), and those versions can no longer be rebuilt at
all — their floating dependency ranges now resolve to @deepseek-ai packages
that have moved past them, so they fail the boot smoke test ~67s in. Step 8
then failed the run for them, and every 6-hour run went red for ~13 minutes of
builds that could never succeed. A green run meant nothing, and a red one no
longer pointed at anything real.

Split the plan into two kinds of build and treat them differently:

  NEW     - the registry has no image for this version. This is an upstream
            release that has not shipped here yet, which is the whole job.
  REFRESH - the registry already has the image; it is only being moved onto a
            newer pnpm/base/recipe. Best-effort.

* Plan NEW first so a new release lands in the first minutes of the run
  instead of queueing behind refreshes that may all be unbuildable.
* A failed NEW version is recorded in a `missing` output and later reported
  by the final gate; a failed REFRESH stays a warning and keeps its image.
* The final gate now fails only on `missing` or on a current version with no
  image in the registry — a missed upstream release is still visible and red.

Also removes two early aborts that silently skipped the prune and the release
publish whenever the current version failed to build: step 4 no longer exits
after a failed build, and the :latest step records its miss instead of
exiting. Publishing now always happens, so the releases for the versions that
did ship are in place even when the run legitimately ends red.
2026-09-28 14:33:27 -04:00

392 lines
21 KiB
YAML

name: Sync upstream and build images
# Mirrors the official DeepSeek Harness repository and ships one container
# image per upstream release, with this repo's fixes applied:
# 1. Mirror upstream `master` and every `dsh-v*` tag — same names, same
# commits, same dates as https://github.com/deepseek-ai/deepseek-harness/tags.
# 2. For every upstream version that is also published on npm (from
# MIN_VERSION up): build the image if the registry lacks it, or rebuild it
# in place when our recipe (Dockerfile, entrypoint, doctor, wrapper, bind
# patch), pnpm, or the node base image changed since it was built. One image
# tag per version — <version>, nothing else.
# 3. Point `:latest` at the newest release-candidate version.
# 4. Publish one Gitea release per built version on the mirrored upstream
# tag, titled like upstream, carrying upstream's own notes plus a short
# image footer. Updated in place when the image is rebuilt.
# 5. A version that fails to build or push fails the run, but only when the
# registry has never had it: a missed upstream release must be visible,
# not a warning in a green run. A *refresh* rebuild — the registry
# already has an image for that version and is only being moved onto a
# newer pnpm/base/recipe — is best-effort: it warns, leaves the existing
# image alone, and cannot turn the run red. Old versions cannot always
# be rebuilt any more, because their floating dependency ranges now
# resolve to packages that have moved past them, and a doomed 0.1.x
# refresh must not hide the 0.2.x release that did ship. Every failure
# stays listed in the run either way.
#
# Images are pushed with regctl, in 50 MB chunks. The registry sits behind
# Cloudflare, which rejects any request body over 100 MB; `docker push` sends a
# layer as one request, and dsh's node_modules layer passed that mark with
# 0.1.6-alpha.2 (413 Payload Too Large).
on:
schedule:
- cron: '17 */6 * * *'
workflow_dispatch:
inputs:
rebuild:
description: Force-rebuild these versions (space-separated) or "all"
required: false
default: ''
push:
branches: [main]
paths:
- Dockerfile
- docker-entrypoint.sh
- dsh-workspace-doctor.mjs
- run-as-runtime-user.sh
- web.cordis.patch.yml
- .gitea/workflows/sync-upstream.yml
- .gitea/scripts/publish-releases.js
env:
UPSTREAM: https://github.com/deepseek-ai/deepseek-harness.git
REGISTRY: gitea.milesward.dev
IMAGE: mward4/deepseek-harness
BASE_IMAGE: node:22-bookworm-slim
# Oldest upstream version this project ships an image for.
MIN_VERSION: 0.1.1-rc.2
# Upstream versions that cannot be built from npm on this base image, with
# the reason, so the run neither retries them every six hours nor warns:
# 0.1.3-alpha.2 — depends on the native module fs-ext, which needs a C
# toolchain (node-gyp) the slim runtime image lacks.
SKIP_VERSIONS: 0.1.3-alpha.2
jobs:
sync-and-build:
runs-on: ubuntu-latest
steps:
- name: Checkout CI branch
uses: actions/checkout@v4
with:
ref: main
# Static binary, pinned by version and checksum. Used to push images in
# chunks (see the header) and to prune untagged manifests.
- name: Install regctl
env:
REGCTL_VERSION: v0.11.6
REGCTL_SHA256: 8e0e62a497fcdb8048d18aa927a139613176ba0531f412bc541044e28f9856bd
run: |
set -eu
curl -fsSL -o /usr/local/bin/regctl "https://github.com/regclient/regclient/releases/download/${REGCTL_VERSION}/regctl-linux-amd64"
echo "${REGCTL_SHA256} /usr/local/bin/regctl" | sha256sum -c -
chmod 0755 /usr/local/bin/regctl
regctl version
- name: Mirror upstream master and dsh-v* tags
env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -eux
git clone --bare --single-branch --branch master "$UPSTREAM" /tmp/upstream
cd /tmp/upstream
git fetch -q origin '+refs/tags/dsh-v*:refs/tags/dsh-v*'
git remote add mirror "https://oauth2:${TOKEN}@${REGISTRY}/${IMAGE}.git"
git push mirror '+refs/heads/master:refs/heads/master' '+refs/tags/dsh-v*:refs/tags/dsh-v*'
git tag -l 'dsh-v*' | sed 's/^dsh-v//' > /tmp/upstream-versions
echo "upstream versions: $(tr '\n' ' ' < /tmp/upstream-versions)"
- name: Plan builds
id: plan
env:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
OWNER: ${{ github.repository_owner }}
REBUILD: ${{ inputs.rebuild }}
run: |
set -eu
PNPM_LATEST=$(npm view pnpm dist-tags.latest)
npm view @deepseek-ai/dsh versions --json | grep -o '"[0-9][^"]*"' | tr -d '"' > /tmp/npm-versions
docker pull -q "$BASE_IMAGE" > /dev/null
BASE_DIGEST=$(docker inspect --format '{{index .RepoDigests 0}}' "$BASE_IMAGE")
RECIPE_HASH=$(cat Dockerfile docker-entrypoint.sh dsh-workspace-doctor.mjs run-as-runtime-user.sh web.cordis.patch.yml | sha256sum | cut -c1-12)
WANT="${PNPM_LATEST}|${BASE_DIGEST}|${RECIPE_HASH}"
echo "pnpm=$PNPM_LATEST base=$BASE_DIGEST recipe=$RECIPE_HASH rebuild='$REBUILD'"
ACCEPT='application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json,application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json'
tag_exists() {
[ "$(curl -s -o /dev/null -w '%{http_code}' -u "${OWNER}:${TOKEN}" \
-H "Accept: $ACCEPT" "https://${REGISTRY}/v2/${IMAGE}/manifests/$1")" = 200 ]
}
fetch_manifest() { curl -s -u "${OWNER}:${TOKEN}" -H "Accept: $ACCEPT" "https://${REGISTRY}/v2/${IMAGE}/manifests/$1"; }
first_digest() { printf '%s' "$1" | grep -o '"digest": *"sha256:[a-f0-9]*"' | head -1 | grep -o 'sha256:[a-f0-9]*'; }
# Provenance labels of an image ("pnpm|base|recipe"); follows a
# multi-arch index to its first platform manifest.
tag_toolchain() {
MAN=$(fetch_manifest "$1") || return 0
if printf '%s' "$MAN" | grep -q '"manifests"'; then
CHILD=$(first_digest "$MAN"); [ -n "$CHILD" ] || return 0
MAN=$(fetch_manifest "$CHILD") || return 0
fi
CFG=$(first_digest "$MAN") || return 0
[ -n "$CFG" ] || return 0
BLOB=$(curl -s -u "${OWNER}:${TOKEN}" "https://${REGISTRY}/v2/${IMAGE}/blobs/$CFG") || return 0
P=$(printf '%s' "$BLOB" | grep -o '"dev.milesward.pnpm-version":"[^"]*"' | cut -d'"' -f4 || true)
B=$(printf '%s' "$BLOB" | grep -o '"dev.milesward.base-digest":"[^"]*"' | cut -d'"' -f4 || true)
R=$(printf '%s' "$BLOB" | grep -o '"dev.milesward.recipe-hash":"[^"]*"' | cut -d'"' -f4 || true)
echo "${P}|${B}|${R}"
}
# v1 >= v2 in version order?
at_least() { [ "$(printf '%s\n%s\n' "$2" "$1" | sort -V | head -1)" = "$2" ]; }
forced() { case " $REBUILD " in *" all "*|*" $1 "*) return 0;; *) return 1;; esac; }
# Candidates: upstream tags that npm actually published, from MIN_VERSION up.
CANDIDATES=""
while read -r V; do
[ -n "$V" ] || continue
grep -qx "$V" /tmp/npm-versions || { echo "skip $V: not on npm"; continue; }
at_least "$V" "$MIN_VERSION" || continue
case " $SKIP_VERSIONS " in *" $V "*) echo "skip $V: listed in SKIP_VERSIONS"; continue;; esac
CANDIDATES="$CANDIDATES $V"
done < <(sort -V /tmp/upstream-versions)
echo "candidates:$CANDIDATES"
# Two kinds of build, and they do not matter equally. NEW is a
# version the registry has never had — an upstream release that has
# not shipped here yet, which is the whole point of this job.
# REFRESH is an already-published version being moved onto newer
# toolchain. Plan NEW first, so a new release lands in the first
# minutes of the run instead of queueing behind refreshes that may
# all be unbuildable.
NEW=""
REFRESH=""
for V in $CANDIDATES; do
if ! tag_exists "$V"; then
echo "$V: missing -> build"; NEW="$NEW $V"
elif forced "$V"; then
echo "$V: rebuild requested"; REFRESH="$REFRESH $V"
else
HAVE=$(tag_toolchain "$V")
if [ "$HAVE" != "$WANT" ]; then echo "$V: stale ($HAVE) -> rebuild"; REFRESH="$REFRESH $V"; fi
fi
done
BUILDS="$NEW $REFRESH"
# Newest release candidate = what :latest points at.
CURRENT=$(printf '%s\n' $CANDIDATES | grep -v alpha | sort -V | tail -1)
echo "new (must ship):$NEW"
echo "refresh (best-effort):$REFRESH"
echo "planned builds:$BUILDS"
echo "current (:latest): $CURRENT"
{
echo "builds=${BUILDS}"
echo "new=${NEW}"
echo "current=${CURRENT}"
echo "pnpm=${PNPM_LATEST}"
echo "base_digest=${BASE_DIGEST}"
echo "recipe_hash=${RECIPE_HASH}"
} >> "$GITHUB_OUTPUT"
- name: Build, smoke-test, and push
id: build
if: steps.plan.outputs.builds != ''
env:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
PNPM_VERSION: ${{ steps.plan.outputs.pnpm }}
BASE_DIGEST: ${{ steps.plan.outputs.base_digest }}
RECIPE_HASH: ${{ steps.plan.outputs.recipe_hash }}
CI_RUN: ${{ github.run_number }}
run: |
set -u
echo "$TOKEN" | regctl registry login "$REGISTRY" -u "${{ github.repository_owner }}" --pass-stdin
# Blobs above 50 MB go up in 50 MB chunks: half of Cloudflare's cap.
regctl registry set "$REGISTRY" --blob-chunk 50000000 --blob-max 50000000
GIT_SHA=$(git rev-parse HEAD)
BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ)
SMOKE="dsh-smoke-${{ github.run_id }}"
# Build one version; smoke-test on a fresh root-owned volume (what a
# new Unraid appdata dir looks like); push only if it passes.
# 200 = pre-token versions, 303/401 = 0.1.2+ launch-token gate.
build_one() (
set -eux
V="$1"; REF="${REGISTRY}/${IMAGE}:${V}"
docker build \
--build-arg "DSH_VERSION=${V}" \
--build-arg "PNPM_VERSION=${PNPM_VERSION}" \
--build-arg "BASE_DIGEST=${BASE_DIGEST}" \
--build-arg "RECIPE_HASH=${RECIPE_HASH}" \
--build-arg "GIT_SHA=${GIT_SHA}" \
--build-arg "CI_RUN=${CI_RUN}" \
--build-arg "BUILD_DATE=${BUILD_DATE}" \
-t "$REF" .
docker rm -f "$SMOKE" 2>/dev/null || true
docker volume rm -f "$SMOKE" 2>/dev/null || true
docker run -d --name "$SMOKE" --mount "type=volume,src=$SMOKE,dst=/home/node/.dsh" "$REF"
sleep 3
[ "$(docker exec "$SMOKE" dsh --version)" = "$V" ]
[ "$(docker exec "$SMOKE" pnpm --version)" = "$PNPM_VERSION" ]
[ "$(docker exec "$SMOKE" awk '/^Uid:/{print $2}' /proc/1/status)" = "1000" ]
# The workspace doctor must be installed, executable and able to read
# a state volume. --help is the strict presence check; --check may
# legitimately answer 1 (something to report) on a fresh volume, so
# only 2 — the tool itself failed — is a build failure.
docker exec "$SMOKE" dsh-workspace-doctor.mjs --help > /dev/null
DOCTOR_RC=0
docker exec "$SMOKE" dsh-workspace-doctor.mjs --check > /dev/null || DOCTOR_RC=$?
[ "$DOCTOR_RC" -le 1 ]
# ...and the entrypoint must actually invoke it. The doctor prints a
# "dsh-workspace-doctor:" line when it runs; the entrypoint logs a
# "workspace doctor:" line only when it could not. A direct
# `docker exec` above sails past a wrong path or a lost exec bit, so
# the boot log, not the binary, is what proves the wiring.
if ! docker logs "$SMOKE" 2>&1 | grep -q "^dsh-workspace-doctor:"; then
echo "the workspace doctor did not run at boot"
docker logs "$SMOKE" | tail -30
exit 1
fi
if docker logs "$SMOKE" 2>&1 | grep -q "workspace doctor:"; then
echo "the entrypoint could not run the workspace doctor"
docker logs "$SMOKE" | tail -30
exit 1
fi
OK=""
for i in $(seq 1 30); do
if docker exec "$SMOKE" node -e '
const os = require("os");
const nic = Object.values(os.networkInterfaces()).flat()
.find(i => i && i.family === "IPv4" && !i.internal);
fetch("http://" + nic.address + ":3080/").then(r => {
process.exit([200, 303, 401].includes(r.status) ? 0 : 1);
}).catch(() => process.exit(2));'
then OK=1; break; fi
sleep 2
done
if [ -z "$OK" ]; then docker logs "$SMOKE" | tail -40; exit 1; fi
docker rm -f "$SMOKE"; docker volume rm -f "$SMOKE" || true
# Export from the daemon and push with regctl (chunked). A daemon
# with the containerd store exports an index with an attestation
# manifest; copy only the image so the tag stays a plain manifest.
rm -rf /tmp/export; mkdir -p /tmp/export
docker save "$REF" -o /tmp/export/image.tar
regctl image import "ocidir:///tmp/export/oci:${V}" /tmp/export/image.tar
# A classic (non-containerd) daemon exports layers uncompressed, and
# regctl pushes what it is given: without this, dsh's 500 MB raw
# layer would reach the registry and every pull would be 4x larger.
regctl image mod "ocidir:///tmp/export/oci:${V}" --layer-compress gzip --replace
PLATFORM=""
if regctl manifest get "ocidir:///tmp/export/oci:${V}" --format raw-body | grep -q '"manifests"'; then PLATFORM="--platform linux/amd64"; fi
regctl image copy $PLATFORM "ocidir:///tmp/export/oci:${V}" "$REF"
rm -rf /tmp/export
regctl manifest head "$REF" > /dev/null
)
FAILED=""
MISSING=""
for V in ${{ steps.plan.outputs.builds }}; do
if build_one "$V"; then
echo "::notice::built ${IMAGE}:${V}"
else
docker rm -f "$SMOKE" 2>/dev/null || true; docker volume rm -f "$SMOKE" 2>/dev/null || true
echo "::warning::${V} did not build or push; any existing image for it is left untouched"
FAILED="$FAILED $V"
# Missing from the registry AND failed to build = an upstream
# release we failed to ship. Record it for the final gate.
case " ${{ steps.plan.outputs.new }} " in *" $V "*) MISSING="$MISSING $V";; esac
fi
done
echo "failed=${FAILED}" >> "$GITHUB_OUTPUT"
echo "missing=${MISSING}" >> "$GITHUB_OUTPUT"
# Deliberately no exit here. Steps 5-7 still have to run so the
# releases for every version that did ship are in place even when
# this run ends red; the single red/green decision belongs to the
# last step. Aborting here is what used to leave a new release
# unpublished.
[ -z "$FAILED" ] || echo "failed versions:$FAILED"
# `:latest` = the newest release-candidate version. A manifest copy
# through the registry API: identical digest, nothing rebuilt.
- name: Point :latest at the current version
id: latest
env:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
OWNER: ${{ github.repository_owner }}
CURRENT: ${{ steps.plan.outputs.current }}
run: |
set -eu
[ -n "$CURRENT" ] || { echo "no current version"; exit 0; }
ACCEPT='application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json,application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json'
digest_of() { curl -sI -u "${OWNER}:${TOKEN}" -H "Accept: $ACCEPT" "https://${REGISTRY}/v2/${IMAGE}/manifests/$1" | grep -i '^docker-content-digest' | awk '{print $2}' | tr -d '\r'; }
WANT=$(digest_of "$CURRENT")
# Do not exit non-zero here either: that would skip both the prune
# and the release publish. Record the miss, let the gate decide.
[ -n "$WANT" ] || { echo "::error::${CURRENT} has no image in the registry, so :latest cannot be moved"; echo "latest_failed=1" >> "$GITHUB_OUTPUT"; exit 0; }
HAVE=$(digest_of latest || true)
echo "latest -> ${CURRENT} (${WANT}); currently ${HAVE:-unset}"
[ "$WANT" != "$HAVE" ] || { echo "latest already current"; exit 0; }
curl -s -u "${OWNER}:${TOKEN}" -H "Accept: $ACCEPT" -D /tmp/hdr -o /tmp/manifest "https://${REGISTRY}/v2/${IMAGE}/manifests/${CURRENT}"
CT=$(grep -i '^content-type' /tmp/hdr | awk '{print $2}' | tr -d '\r')
CODE=$(curl -s -o /tmp/put.out -w '%{http_code}' -X PUT -u "${OWNER}:${TOKEN}" -H "Content-Type: ${CT}" \
--data-binary @/tmp/manifest "https://${REGISTRY}/v2/${IMAGE}/manifests/latest")
[ "$CODE" = 201 ] || { echo "::error::retag failed: HTTP $CODE"; cat /tmp/put.out; exit 1; }
echo "latest now -> ${CURRENT} ($(digest_of latest))"
# Every rebuild of a version tag leaves its previous manifest behind as
# an untagged package version (a `sha256:` row on the Packages page that
# keeps its layers on disk). Remove those, keeping any manifest that a
# tagged index still points at.
- name: Prune untagged manifests
env:
TOKEN: ${{ secrets.REGISTRY_TOKEN }}
OWNER: ${{ github.repository_owner }}
run: |
set -eu
API="https://${REGISTRY}/api/v1"
NAME="${IMAGE#*/}"
LIST=$(curl -s -u "${OWNER}:${TOKEN}" "${API}/packages/${OWNER}?type=container&q=${NAME}&limit=100")
VERSIONS=$(printf '%s' "$LIST" | grep -o '"version":"[^"]*"' | cut -d'"' -f4 | sort -u)
KEEP=" "
for T in $VERSIONS; do
case "$T" in sha256:*) continue;; esac
MAN=$(regctl manifest get "${REGISTRY}/${IMAGE}:${T}" --format raw-body 2>/dev/null || true)
case "$MAN" in *'"manifests"'*) KEEP="$KEEP$(printf '%s' "$MAN" | grep -o '"digest":"sha256:[a-f0-9]*"' | cut -d'"' -f4 | tr '\n' ' ') ";; esac
done
N=0
for D in $VERSIONS; do
case "$D" in sha256:*) ;; *) continue;; esac
case "$KEEP" in *" $D "*) echo "keep $D (referenced by a tagged index)"; continue;; esac
CODE=$(curl -s -o /dev/null -w '%{http_code}' -X DELETE -u "${OWNER}:${TOKEN}" "${API}/packages/${OWNER}/container/${NAME}/${D/:/%3A}")
echo "pruned $D (HTTP $CODE)"; N=$((N+1))
done
echo "pruned $N untagged manifest(s)"
# One release per built version, on the mirrored upstream tag, titled
# like upstream and carrying upstream's notes; updated in place when the
# image is rebuilt. Runs on the image base so the runner needs nothing.
- name: Publish releases
env:
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REG_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
OWNER: ${{ github.repository_owner }}
REPO: ${{ github.repository }}
RUN_NUMBER: ${{ github.run_number }}
run: |
set -eu
MAIN_SHA=$(git rev-parse HEAD)
docker run --rm -i \
-e "GITEA=https://${REGISTRY}" -e OWNER -e REPO -e IMAGE -e GITEA_TOKEN -e REG_TOKEN \
-e "MAIN_SHA=${MAIN_SHA}" -e RUN_NUMBER \
"$BASE_IMAGE" node - < .gitea/scripts/publish-releases.js
# Last, so :latest and the releases for everything that did ship are
# already in place before the run turns red — and the only thing that
# turns it red is a version this registry has never published. Refreshes
# of versions we already ship stay warnings.
- name: Fail the run if a new version did not ship
if: steps.build.outputs.missing != '' || steps.latest.outputs.latest_failed != ''
run: |
[ -z "${{ steps.build.outputs.missing }}" ] || \
echo "::error::never-published versions did not build or push:${{ steps.build.outputs.missing }}"
[ "${{ steps.latest.outputs.latest_failed }}" != "1" ] || \
echo "::error::${{ steps.plan.outputs.current }} has no image in the registry, so :latest was not moved"
exit 1