Add CI: six-hourly dependency checks, and a tag-driven image release
Build and release image / build (push) Failing after 3m5s

Two workflows, both aimed at the same problem: this container runs unattended
on someone's Unraid box, so the things that keep it correct have to happen
without anyone remembering to do them.

## dependency-check.yml

Runs every six hours and reports everything that has drifted in one place:
crates from Cargo.lock, the Rust pin in rust-toolchain.toml and the Dockerfile,
and the Dockerfile's base images. The result lands in a single issue titled
"Dependency updates" that is rewritten in place rather than commented on, and
is closed automatically when nothing is out of date — an open issue that never
changes is one people stop reading.

Nothing is upgraded automatically. `cargo update` on a schedule would put
unreviewed code into a service holding the operator's second factor, so the
report separates what `cargo update` covers from what crosses a breaking
boundary, and leaves the decision alone.

`.gitea/scripts/dependency-report.mjs` does the work and is deliberately
standalone, so the same file serves the sibling Rust repos. Two details worth
knowing:

  * Crates are read from the sparse index, not the crates.io API. The API rate
    limits around 130 lookups and answers 429, which silently turned most of
    the first report into "could not check"; the index is one request per crate
    with no limit.
  * The pre-1.0 rule is applied properly. semver treats `0.x.y` as unstable, so
    `argon2 0.5 -> 0.6` breaks exactly as hard as `1.5 -> 2.0`. Getting this
    wrong labelled half the RustCrypto family "minor or patch" when `cargo
    update` will not touch any of them.

## release.yml

Tag `vX.Y.Z` and this builds the image, starts it on a fresh root-owned volume,
waits for /healthz, checks the binary reports the tagged version, pushes to the
Gitea registry, and only then creates the release with the image digest in the
notes. Order matters: a release announcing an image that isn't in the registry
is worse than no release, and a broken build must not overwrite a tag that
already works.

It also asserts the tag and Cargo.toml agree, and rebuilds `:latest` on a
Dockerfile change without touching releases. Hand-written notes come from
docs/releases/<version>.md.

Requires a REGISTRY_TOKEN secret with write:package. Note that Gitea reserves
the `GITEA_` prefix for secret names and rejects it, which is why the secret is
not called GITEA_TOKEN.
This commit is contained in:
DSH Agent committed 2026-09-20 02:48:27 -04:00
1 parent b21ade05eb
commit 60348b8133
4 files changed
+1006

No files matched your search

+516
View File
@@ -0,0 +1,516 @@
//! Report every dependency that has a newer release than the one in use.
//!
//! Runs from `.gitea/workflows/dependency-check.yml` on a six-hour schedule.
//! Prints a Markdown report on stdout and exits 0 whether or not anything is
//! out of date — "up to date" is a normal outcome, not a failure. The workflow
//! decides what to do with the report; this file only gathers facts.
//!
//! Three things are checked, because all three go stale independently and none
//! of them is visible from the others:
//!
//! * **Crates** — every locked dependency in `Cargo.lock` against crates.io.
//! * **The Rust pin** — `rust-toolchain.toml`'s channel against the latest
//! stable release, and the Dockerfile's `RUST_VERSION` against both, since
//! those two drifting apart is how a local build and a shipped image end up
//! on different compilers.
//! * **Base images** — the Dockerfile's `FROM` tags against their upstream
//! registries.
//!
//! Network only: no crates are downloaded, nothing is built, and the working
//! tree is never modified.
import { readFileSync, existsSync } from "node:fs";
import { execFileSync } from "node:child_process";
const UA = "dependency-check (+https://gitea.milesward.dev/mward4)";
// ── version comparison ──────────────────────────────────────────────────────
/// Parse `1.2.3`, `1.2`, `1.2.3-alpha.1`, and `=1.2.3` into a comparable
/// tuple. Returns null for anything that isn't a version we understand, so an
/// unparseable requirement is reported as "unknown" rather than silently
/// sorting to the top.
function parseVersion(text) {
const m = /^[=~^]?\s*v?(\d+)(?:\.(\d+))?(?:\.(\d+))?(?:[-+](.*))?$/.exec(text.trim());
if (!m) return null;
return {
major: Number(m[1]),
minor: Number(m[2] ?? 0),
patch: Number(m[3] ?? 0),
pre: m[4] ?? "",
};
}
/// Order two parsed versions. A pre-release sorts below its release, so
/// `2.0.0-rc.1` never displaces a stable `1.9.0` as "the newest".
function compare(a, b) {
for (const key of ["major", "minor", "patch"]) {
if (a[key] !== b[key]) return a[key] < b[key] ? -1 : 1;
}
if (a.pre === b.pre) return 0;
if (!a.pre) return 1;
if (!b.pre) return -1;
return a.pre < b.pre ? -1 : 1;
}
/// How a version difference is allowed to be taken.
///
/// The pre-1.0 rule is the one that matters here: semver treats `0.x.y` as
/// unstable, so `0.5 → 0.6` breaks exactly as hard as `1.5 → 2.0`. Getting this
/// wrong would label half the RustCrypto family "minor or patch" when in fact
/// `cargo update` will refuse every one of them.
function bumpKind(from, to) {
if (to.major !== from.major) return "breaking";
if (from.major === 0 && to.minor !== from.minor) return "breaking";
return "compatible";
}
/// The newest release reached without crossing a breaking boundary — what
/// `cargo update` will move to on its own.
function bestCompatible(from, versions) {
let best = null;
for (const raw of versions) {
const v = parseVersion(raw);
// `bumpKind` is checked before comparing against `from`: `0.14.7` is not
// semver-compatible with `1.0.0` even though it is numerically lower, and
// the naive `compare > 0` test would let `0.14 → 1.4` through as routine.
if (!v || bumpKind(from, v) !== "compatible" || compare(v, from) <= 0) continue;
if (!best || compare(v, best) > 0) best = v;
}
return best;
}
/// The newest release of any kind, which for a breaking bump is what a human
/// would have to adopt.
function bestOverall(from, versions) {
let best = null;
for (const raw of versions) {
const v = parseVersion(raw);
if (!v || compare(v, from) <= 0) continue;
if (!best || compare(v, best) > 0) best = v;
}
return best;
}
const fmt = (v) => (v ? `${v.major}.${v.minor}.${v.patch}${v.pre ? `-${v.pre}` : ""}` : "?");
// ── inputs ──────────────────────────────────────────────────────────────────
/// Resolve the dependency graph with cargo itself.
///
/// `--locked` rather than a hand-rolled Cargo.lock parse: it fails instead of
/// silently rewriting the lockfile, which matters because this runs on a
/// schedule and must never mutate the repository it is reporting on. Cargo.toml
/// is still read directly below for the declared requirement, since metadata
/// reports the resolved version rather than the constraint.
function cargoMetadata() {
let raw;
try {
raw = execFileSync("cargo", ["metadata", "--format-version", "1", "--locked"], {
encoding: "utf8",
maxBuffer: 64 * 1024 * 1024,
stdio: ["ignore", "pipe", "pipe"],
});
} catch (e) {
// A cargo failure here means no report at all. Say why in one line rather
// than letting a raw stack trace stand in for an explanation.
const detail = (e.stderr || e.message || "").toString().trim().split("\n")[0];
throw new Error(`cargo metadata failed: ${detail}`);
}
return JSON.parse(raw);
}
function declaredRequirements() {
// Handles the `dep = "1.2"` and `dep = { version = "1.2", ... }` forms and
// nothing else, which is every dependency this workspace declares.
const text = readFileSync("Cargo.toml", "utf8");
const out = new Map();
let section = "";
for (const line of text.split("\n")) {
const header = /^\[([^\]]+)\]/.exec(line);
if (header) {
section = header[1];
continue;
}
if (!/dependencies/.test(section)) continue;
const m = /^\s*([A-Za-z0-9_-]+)\s*=\s*(.+?)\s*$/.exec(line);
if (!m) continue;
const [, name, value] = m;
const quoted = /^"([^"]+)"/.exec(value);
if (quoted) out.set(name, quoted[1]);
else {
const v = /version\s*=\s*"([^"]+)"/.exec(value);
if (v) out.set(name, v[1]);
}
}
return out;
}
// ── external lookups ────────────────────────────────────────────────────────
/// The latest stable Rust, from the release channel manifest.
///
/// Read from the sparse index rather than the crates.io API. The API is rate
/// limited and answers 429 under a burst of ~130 lookups, which silently
/// turned most of this report into "could not check" on the first run; the
/// index is the same data, one request per crate, no limit and no user agent
/// requirement.
async function publishedVersions(name) {
// The index shards by name length and case: 1-char names live under "1",
// 2-char under "2", 3-char under "3/<first letter>", and everything else
// under the first two characters. Names are lowercased on the way in.
const lower = name.toLowerCase();
const prefix =
lower.length === 1
? `1/${lower}`
: lower.length === 2
? `2/${lower}`
: lower.length === 3
? `3/${lower[0]}/${lower}`
: `${lower.slice(0, 2)}/${lower.slice(2, 4)}/${lower}`;
try {
const res = await fetch(`https://index.crates.io/${prefix}`, {
headers: { "user-agent": UA },
});
// The index answers 404 for a crate that does not exist and 410 for one
// that was removed; both mean "nothing to compare against".
if (!res.ok) return null;
return (await res.text())
.split("\n")
.filter(Boolean)
.map((line) => JSON.parse(line))
.filter((entry) => !entry.yanked && !entry.vers.includes("-"))
.map((entry) => entry.vers);
} catch {
return null;
}
}
/// The latest stable Rust, from the release channel manifest.
async function latestRust() {
try {
const res = await fetch("https://static.rust-lang.org/dist/channel-rust-stable.toml", {
headers: { "user-agent": UA },
});
if (!res.ok) return null;
const m = /\[pkg\.rust\]\nversion = "([^"]+)"/.exec(await res.text());
// The manifest value carries the commit: "1.98.1 (48a229cea 2026-09-01)".
// Only the number is comparable to the Dockerfile's RUST_VERSION.
return m ? m[1].split(/\s+/)[0] : null;
} catch {
return null;
}
}
/// Tags for a Docker Hub repository, or null if it can't be read.
async function dockerTags(repo) {
const [ns, name] = repo.includes("/") ? repo.split("/") : ["library", repo];
try {
const res = await fetch(`https://hub.docker.com/v2/repositories/${ns}/${name}/tags?page_size=100`, {
headers: { "user-agent": UA },
});
if (!res.ok) return null;
const data = await res.json();
return data.results.map((t) => t.name);
} catch {
return null;
}
}
// ── checks ──────────────────────────────────────────────────────────────────
async function checkCrates() {
const meta = cargoMetadata();
const requirements = declaredRequirements();
// One row per crate actually compiled into the workspace. Dev-dependencies
// and build-dependencies are real dependencies too, so they are included —
// a stale build script is still a stale build script.
const seen = new Map();
for (const pkg of meta.packages) {
if (meta.workspace_members.includes(pkg.id)) continue;
seen.set(pkg.name, { from: pkg.version, kind: pkg.source ? "registry" : "path" });
}
const rows = [];
for (const [name, info] of [...seen].sort((a, b) => a[0].localeCompare(b[0]))) {
if (info.kind === "path") continue;
// Direct means "named in Cargo.toml", which is the only set the author can
// act on. The rest move when whatever pulls them in moves, so they are
// reported but never counted as work.
const direct = requirements.has(name);
const current = parseVersion(info.from);
const versions = await publishedVersions(name);
if (!current || !versions) {
rows.push({ name, from: info.from, to: null, kind: "unknown", direct, note: "could not check" });
continue;
}
const compatible = bestCompatible(current, versions);
const overall = bestOverall(current, versions);
if (!overall) continue; // already newest
// `compatible` is what `cargo update` will do unaided. If the newest
// release is further out than that, the gap is a breaking change someone
// has to adopt deliberately.
const kind = compatible ? "compatible" : bumpKind(current, overall);
rows.push({
name,
from: info.from,
to: fmt(overall),
kind,
direct,
note: kind === "compatible" ? "`cargo update`" : "**breaking — needs review**",
});
}
return rows;
}
function rustPin() {
const toolchain = existsSync("rust-toolchain.toml")
? readFileSync("rust-toolchain.toml", "utf8")
: "";
const channel = /channel\s*=\s*"([^"]+)"/.exec(toolchain)?.[1] ?? null;
// The image bakes its own toolchain, so this is the number that decides what
// actually compiles the release binary.
const dockerfile = existsSync("deploy/docker/Dockerfile")
? readFileSync("deploy/docker/Dockerfile", "utf8")
: existsSync("Dockerfile")
? readFileSync("Dockerfile", "utf8")
: "";
const image = /ARG\s+RUST_VERSION=([0-9.]+)/.exec(dockerfile)?.[1] ?? null;
// The number `cargo` enforces as the minimum supported toolchain.
const manifest = readFileSync("Cargo.toml", "utf8");
const msrv = /rust-version\s*=\s*"([^"]+)"/.exec(manifest)?.[1] ?? null;
return { channel, image, msrv };
}
/// Every `FROM` in the Dockerfile, with the `ARG` substitution applied for the
/// one argument this build actually parameterises.
function baseImages() {
const path = existsSync("deploy/docker/Dockerfile")
? "deploy/docker/Dockerfile"
: existsSync("Dockerfile")
? "Dockerfile"
: null;
if (!path) return [];
const text = readFileSync(path, "utf8");
const args = new Map(
[...text.matchAll(/^ARG\s+([A-Za-z0-9_]+)=(\S+)/gm)].map((m) => [m[1], m[2]]),
);
const images = [];
for (const m of text.matchAll(/^FROM\s+(?:--platform=\S+\s+)?(\S+)/gm)) {
let ref = m[1];
// `node:${NODE_VERSION}-slim` and friends.
ref = ref.replace(/\$\{?([A-Za-z0-9_]+)\}?/g, (whole, name) => args.get(name) ?? whole);
if (ref.includes("$")) continue; // still unresolved; skip rather than guess
const [repo, tag = "latest"] = ref.split(":");
images.push({ repo, tag });
}
return images;
}
async function checkRustToolchain() {
const pin = rustPin();
const latest = await latestRust();
return { ...pin, latest };
}
/// The newest tag that keeps the variant suffix the Dockerfile asked for.
///
/// `rust:1.95-bookworm` and `node:22-bookworm-slim` are not interchangeable
/// with their plain counterparts, so a tag is only a candidate if it ends in
/// exactly the suffix already in use.
function bestTagFor(currentTag, tags) {
const m = /^([\d.]+)(-.*)?$/.exec(currentTag);
if (!m) return null;
const [, , suffix = ""] = m;
let best = null;
for (const tag of tags) {
if (suffix) {
if (!tag.endsWith(suffix)) continue;
} else if (!/^[\d.]+$/.test(tag)) {
continue;
}
const v = parseVersion(suffix ? tag.slice(0, -suffix.length) : tag);
// Date-stamped tags like `alpine:20260805` parse as versions but are edge
// or snapshot builds, not releases. No official image we track is anywhere
// near a three-digit major.
if (v && !suffix && v.major >= 100) continue;
if (v && (!best || compare(v, best) > 0)) best = v;
}
return best;
}
async function checkImages() {
const rows = [];
for (const { repo, tag } of baseImages()) {
const tags = await dockerTags(repo);
if (!tags) {
rows.push({ repo, from: tag, to: null, note: "could not check" });
continue;
}
const parsed = parseVersion(tag);
const best = bestTagFor(tag, tags);
if (!parsed || !best) {
rows.push({ repo, from: tag, to: null, note: "tag is not a version" });
continue;
}
// A base image major bump is a whole new distro release; same rule as a
// crate, so it is reported but flagged rather than treated as routine.
const bump = bumpKind(parsed, best);
rows.push({
repo,
from: tag,
to: fmt(best),
note: bump === "compatible" ? "" : "**breaking — needs review**",
});
}
return rows;
}
// ── report ──────────────────────────────────────────────────────────────────
function table(rows, columns) {
const head = `| ${columns.map((c) => c.title).join(" | ")} |`;
const rule = `| ${columns.map(() => "---").join(" | ")} |`;
const body = rows.map((r) => `| ${columns.map((c) => c.get(r)).join(" | ")} |`);
return [head, rule, ...body].join("\n");
}
async function main() {
const [crates, rust, images] = await Promise.all([checkCrates(), checkRustToolchain(), checkImages()]);
// Split by who can act. Only a crate named in Cargo.toml can be moved by
// hand; everything else follows whichever direct dependency pulls it in, so
// listing those as tasks would be noise that trains the reader to ignore the
// report.
const direct = crates.filter((r) => r.direct);
const transitive = crates.filter((r) => !r.direct);
const routine = direct.filter((r) => r.kind === "compatible");
const breaking = direct.filter((r) => r.kind === "breaking");
const rustOutdated = Boolean(rust.latest && rust.image && rust.image !== rust.latest);
// Transitive crates held back by a version boundary upstream has not crossed.
// Worth knowing, not worth acting on.
const heldBack = transitive.filter((r) => r.kind === "breaking");
const parts = [];
const actionable = routine.length + breaking.length + images.length + (rustOutdated ? 1 : 0);
parts.push(
actionable === 0
? "Everything you can act on is current."
: `${routine.length} routine bump${routine.length === 1 ? "" : "s"}, ` +
`${breaking.length} breaking change${breaking.length === 1 ? "" : "s"}` +
(images.length ? `, ${images.length} base image${images.length === 1 ? "" : "s"}` : "") +
(rustOutdated ? ", and a newer Rust" : "") +
".",
);
if (heldBack.length) {
parts.push(
`_A further ${heldBack.length} transitive crates have newer releases blocked behind a ` +
"breaking boundary upstream has not crossed yet. They are omitted below because " +
"nothing here can move them._",
);
}
if (routine.length) {
parts.push(
"### Direct dependencies — `cargo update` covers these\n\n" +
table(routine, [
{ title: "Crate", get: (r) => `\`${r.name}\`` },
{ title: "In use", get: (r) => `\`${r.from}\`` },
{ title: "Latest", get: (r) => `\`${r.to}\`` },
]),
);
}
if (breaking.length) {
parts.push(
"### Direct dependencies — breaking, a human has to decide\n\n" +
"These cross a version boundary where semver allows anything to change. " +
"`cargo update` will not touch them: `Cargo.toml` needs editing and the " +
"changelog reading.\n\n" +
table(breaking, [
{ title: "Crate", get: (r) => `\`${r.name}\`` },
{ title: "In use", get: (r) => `\`${r.from}\`` },
{ title: "Latest", get: (r) => `\`${r.to}\`` },
]),
);
}
const rustRows = rust.latest
? [
{
what: "Dockerfile `RUST_VERSION`",
using: rust.image ?? "not pinned",
latest: rust.latest,
note: rustOutdated ? "**the shipped binary is built by an older compiler**" : "current",
},
{
what: "`rust-toolchain.toml` channel",
using: rust.channel ?? "unknown",
latest: rust.latest,
note:
rust.channel === "stable"
? "tracks stable, so it is already current"
: "pinned to a fixed release",
},
]
: [
{
what: "latest stable Rust",
using: rust.image ?? "unknown",
latest: "—",
note: "could not reach static.rust-lang.org",
},
];
parts.push(
"### Rust\n\n" +
table(rustRows, [
{ title: "What", get: (r) => r.what },
{ title: "Declared", get: (r) => `\`${r.using}\`` },
{ title: "Latest", get: (r) => `\`${r.latest}\`` },
{ title: "Note", get: (r) => r.note },
]) +
(rust.msrv ? `\n\nMinimum supported Rust version: \`${rust.msrv}\`.` : ""),
);
if (images.length) {
parts.push(
"### Base images\n\n" +
table(images, [
{ title: "Image", get: (r) => `\`${r.repo}\`` },
{ title: "In use", get: (r) => `\`${r.from}\`` },
{ title: "Latest", get: (r) => `\`${r.to}\`` },
{ title: "Note", get: (r) => r.note },
]),
);
}
parts.push("_Checked every 6 hours by `.gitea/workflows/dependency-check.yml`._");
console.log(parts.join("\n\n"));
// Machine-readable footer, so the workflow never has to parse prose.
console.error(
`::dep-check:: routine=${routine.length} breaking=${breaking.length} ` +
`images=${images.length} rust=${rustOutdated ? 1 : 0}`,
);
}
try {
await main();
} catch (e) {
// Print the reason and exit non-zero. The workflow treats a missing report as
// a broken check rather than a clean bill of health, so this keeps the
// tracking issue open instead of closing it on no evidence.
console.error(`dependency check failed: ${e.message}`);
process.exit(1);
}
+160
View File
@@ -0,0 +1,160 @@
name: Check for dependency updates
# Runs every six hours and reports anything out of date in one place: the
# crates in Cargo.lock, the Rust pin in rust-toolchain.toml and the Dockerfile,
# and the Dockerfile's base images.
#
# Everything lands in ONE issue titled "Dependency updates", edited in place
# rather than commented on, so there is a single current answer instead of a
# pile of history. When nothing is out of date the issue is closed — an empty
# issue left open is a thing people stop reading, which defeats the point.
#
# Nothing is upgraded automatically. `cargo update` on a schedule would land
# unreviewed code in a service that holds the operator's second factor, and a
# breaking bump needs a human regardless.
on:
schedule:
# Every six hours, off the hour so this doesn't compete with an image
# rebuild for a runner.
- cron: '23 */6 * * *'
workflow_dispatch:
# One run at a time. A manual dispatch during a scheduled run would otherwise
# race it and could open the issue twice.
concurrency:
group: dependency-check
cancel-in-progress: false
permissions:
contents: read
issues: write
jobs:
check:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v4
# The workspace is streamed into the container over stdin rather than
# bind-mounted, so this works whether the runner shares a filesystem with
# the daemon or not. The official rust image already carries the
# toolchain, so nothing is downloaded inside.
#
# RUSTUP_TOOLCHAIN overrides rust-toolchain.toml: the image's own
# toolchain is enough to read Cargo.lock, and honouring the file would
# make every run fetch a second compiler for no benefit.
#
# Cargo is deliberately *not* run offline: at least one project here
# depends on a crate published to a private registry, and `--offline`
# turns that into a failed resolution and therefore no report at all.
- name: Generate report
run: |
set -eu
tar cf - --exclude=./.git --exclude=./target . | docker run --rm -i \
-e RUSTUP_TOOLCHAIN=1.95 \
-e CARGO_TERM_COLOR=never \
rust:1.95-bookworm \
sh -ec 'mkdir -p /work && cd /work && tar xf - && exec node .gitea/scripts/dependency-report.mjs' \
> /tmp/report.md 2> /tmp/report.err || {
echo "::error::the dependency report failed to generate"
cat /tmp/report.err
exit 1
}
cat /tmp/report.md
grep -E '::dep-check::' /tmp/report.err || true
# An empty report is a broken check, not a clean bill of health.
# Failing here keeps the tracking issue open rather than closing it on
# the strength of a file that never got written.
test -s /tmp/report.md
grep -oE 'routine=[0-9]+ breaking=[0-9]+ images=[0-9]+ rust=[0-9]+' /tmp/report.err \
> /tmp/counts
cat /tmp/counts
- name: Open, update or close the tracking issue
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
REPO: ${{ github.repository }}
run: |
set -eu
API="https://gitea.milesward.dev/api/v1"
node -e '
const fs = require("fs");
const report = fs.readFileSync("/tmp/report.md", "utf8").trim();
const counts = fs.readFileSync("/tmp/counts", "utf8").trim();
const [routine, breaking, images, rust] = counts.match(/[0-9]+/g).map(Number);
const actionable = routine + breaking + images + rust;
const body =
(actionable === 0
? "Everything tracked is current. Closing this until something changes.\n\n"
: "") +
report +
"\n\n---\n\n" +
"Rewritten by `.gitea/workflows/dependency-check.yml` every 6 hours, so this is " +
"always the latest report rather than a pile of comments.";
fs.writeFileSync("/tmp/state.json", JSON.stringify({ actionable, body }));
'
ACTIONABLE=$(node -e 'process.stdout.write(String(require("/tmp/state.json").actionable))')
echo "actionable=$ACTIONABLE"
# Existing issue by title, any state: a previously closed one is
# exactly the one to update and reopen, and searching all states is
# also what stops a duplicate being created after a close.
#
# The `|| true` matters: `set -e` plus `pipefail` would abort the step
# on a transient API error, and creating a second issue is a far worse
# outcome than missing one run.
NUMBER=$(curl -sf -H "Authorization: token ${GITEA_TOKEN}" \
"${API}/repos/${REPO}/issues?state=all&type=issues&limit=100" \
| node -e '
let s = "";
process.stdin.on("data", d => s += d).on("end", () => {
try {
const issues = JSON.parse(s);
const hit = issues.find(i => i.title === "Dependency updates");
process.stdout.write(hit ? String(hit.number) : "");
} catch { process.stdout.write(""); }
});
' || true)
BODY=$(node -e 'process.stdout.write(JSON.stringify({ body: require("/tmp/state.json").body }))')
if [ -z "$NUMBER" ]; then
if [ "$ACTIONABLE" = "0" ]; then
echo "everything current and no issue open — nothing to do"
exit 0
fi
curl -sf -X POST -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
-d "$(node -e 'process.stdout.write(JSON.stringify({ title: "Dependency updates", body: require("/tmp/state.json").body }))')" \
"${API}/repos/${REPO}/issues" > /dev/null
echo "opened the tracking issue"
exit 0
fi
curl -sf -X PATCH -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
-d "$BODY" "${API}/repos/${REPO}/issues/${NUMBER}" > /dev/null
echo "updated issue #${NUMBER}"
# Reopen when there is work, close when there isn't. Done as a second
# call because the state is not part of the body above.
if [ "$ACTIONABLE" = "0" ]; then WANT=closed; else WANT=open; fi
STATE=$(curl -sf -H "Authorization: token ${GITEA_TOKEN}" \
"${API}/repos/${REPO}/issues/${NUMBER}" \
| node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>process.stdout.write(JSON.parse(s).state))')
if [ "$STATE" != "$WANT" ]; then
curl -sf -X PATCH -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"state\":\"${WANT}\"}" "${API}/repos/${REPO}/issues/${NUMBER}" > /dev/null
echo "issue #${NUMBER}: ${STATE} -> ${WANT}"
fi
+241
View File
@@ -0,0 +1,241 @@
name: Build and release image
# Builds the container image and publishes a Gitea release whenever a version
# tag is pushed:
#
# git tag -a v0.1.3 -m "..." && git push origin v0.1.3
#
# The image is built, started, and checked for a working /healthz before it is
# pushed, so a broken build cannot overwrite a tag that already works. The image
# is pushed before the release is created: a release announcing an image that
# isn't in the registry yet is worse than no release.
#
# Pushing to main with a Dockerfile change rebuilds and repushes the current
# version so :latest tracks the fix, but deliberately does not touch releases.
on:
push:
tags:
- 'v*'
branches: [main]
paths:
- 'deploy/docker/**'
- '.gitea/workflows/release.yml'
- 'docker-compose.yml'
workflow_dispatch:
inputs:
version:
description: 'Version to build (defaults to the version in Cargo.toml)'
required: false
default: ''
concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: write
packages: write
env:
REGISTRY: gitea.milesward.dev
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@v4
# Everything downstream needs the version and the image name, so both are
# resolved once here.
#
# A tag is the source of truth when there is one; otherwise the workspace
# version in Cargo.toml, which is what a main-branch rebuild should ship.
# The two are asserted to agree, because a tag that disagrees with the
# binary's own version is a support conversation waiting to happen.
#
# The image name comes from the repository, lowercased because an OCI
# registry rejects uppercase paths. That derivation is what lets this file
# be copied to a sibling repo unchanged.
- name: Resolve version and image
id: version
run: |
set -eu
MANIFEST=$(grep -m1 '^version' Cargo.toml | sed 's/.*"\(.*\)".*/\1/')
if [ "${{ github.ref_type }}" = "tag" ]; then
TAG="${{ github.ref_name }}"
VERSION="${TAG#v}"
if [ "$VERSION" != "$MANIFEST" ]; then
echo "::error::tag ${TAG} says ${VERSION} but Cargo.toml says ${MANIFEST}"
exit 1
fi
else
VERSION="${{ inputs.version }}"
[ -n "$VERSION" ] || VERSION="$MANIFEST"
fi
OWNER=$(printf '%s' "${{ github.repository_owner }}" | tr '[:upper:]' '[:lower:]')
NAME=$(printf '%s' "${{ github.repository }}" | sed 's#.*/##' | tr '[:upper:]' '[:lower:]')
IMAGE="${OWNER}/${NAME}"
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "image=${IMAGE}" >> "$GITHUB_OUTPUT"
echo "ref=${REGISTRY}/${IMAGE}" >> "$GITHUB_OUTPUT"
echo "building ${REGISTRY}/${IMAGE}:${VERSION}"
# The port the health check below probes. Read from the Dockerfile rather
# than hard-coded so this workflow travels with the repo.
- name: Resolve health port
id: port
run: |
set -eu
PORT=$(grep -E '^EXPOSE ' deploy/docker/Dockerfile | head -1 | awk '{print $2}' | cut -d/ -f1)
[ -n "$PORT" ] || { echo "::error::no EXPOSE in deploy/docker/Dockerfile"; exit 1; }
echo "port=${PORT}" >> "$GITHUB_OUTPUT"
echo "health port ${PORT}"
- name: Build image
run: |
set -eu
docker build \
--pull \
--platform linux/amd64 \
--file deploy/docker/Dockerfile \
--tag "${{ steps.version.outputs.ref }}:${{ steps.version.outputs.version }}" \
--tag "${{ steps.version.outputs.ref }}:latest" \
--build-arg "GIT_SHA=$(git rev-parse HEAD)" \
.
# The image is only as good as its ability to boot. This runs the real
# entrypoint with a real (root-owned, empty) data volume — the state a
# fresh Unraid appdata directory is in — and waits for /healthz.
- name: Smoke test
run: |
set -eu
REF="${{ steps.version.outputs.ref }}:${{ steps.version.outputs.version }}"
PORT="${{ steps.port.outputs.port }}"
NAME="da-smoke-${{ github.run_id }}"
VOL="da-smoke-data-${{ github.run_id }}"
cleanup() { docker rm -f "$NAME" >/dev/null 2>&1 || true; docker volume rm -f "$VOL" >/dev/null 2>&1 || true; }
trap cleanup EXIT
docker run -d --name "$NAME" \
--mount "type=volume,src=${VOL},dst=/var/lib/discord-authenticator" \
"$REF"
OK=""
for _ in $(seq 1 30); do
if docker exec "$NAME" wget -qO- "http://127.0.0.1:${PORT}/healthz" 2>/dev/null | grep -qx ok; then
OK=1
break
fi
sleep 1
done
if [ -z "$OK" ]; then
echo "::error::the image did not become healthy"
docker logs "$NAME" || true
exit 1
fi
# A normal boot must have created the vault and the push state, and
# the binary must report the version we tagged — proof that the tag,
# the Cargo.toml bump and the image all agree.
docker exec "$NAME" sh -ec '
test -f /var/lib/discord-authenticator/vault.key
test -f /var/lib/discord-authenticator/push-state.json
/usr/local/bin/discord-authenticator --version
' | tee /tmp/smoke-version
grep -q "${{ steps.version.outputs.version }}" /tmp/smoke-version \
|| { echo "::error::the image reports a different version than the tag"; exit 1; }
docker logs "$NAME"
# LOGIN uses the repository owner rather than a fixed name: Gitea's
# registry authenticates against the account that owns the token.
- name: Push to the registry
id: push
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
[ -n "${GITEA_TOKEN:-}" ] || {
echo "::error::no REGISTRY_TOKEN secret. Add one under Settings -> Actions -> Secrets"
echo "::error::with the write:package scope, then re-run this workflow."
exit 1
}
echo "$GITEA_TOKEN" | docker login "$REGISTRY" --username "${{ github.repository_owner }}" --password-stdin
docker push "${{ steps.version.outputs.ref }}:${{ steps.version.outputs.version }}"
docker push "${{ steps.version.outputs.ref }}:latest"
docker logout "$REGISTRY" || true
# Record what shipped, so the release body can name the exact digest.
DIGEST=$(docker inspect --format '{{index .RepoDigests 0}}' \
"${{ steps.version.outputs.ref }}:${{ steps.version.outputs.version }}" | sed 's/.*@//')
echo "digest=${DIGEST}" >> "$GITHUB_OUTPUT"
echo "pushed ${{ steps.version.outputs.ref }}:${{ steps.version.outputs.version }} @ ${DIGEST}"
# Releases are created only for tags. A main-branch rebuild ships the same
# version again and must not rewrite the notes of an existing release.
- name: Publish the release
if: github.ref_type == 'tag'
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
REPO: ${{ github.repository }}
VERSION: ${{ steps.version.outputs.version }}
IMAGE: ${{ steps.version.outputs.image }}
DIGEST: ${{ steps.push.outputs.digest }}
run: |
set -eu
API="https://${REGISTRY}/api/v1"
TAG="${{ github.ref_name }}"
# Hand-written notes when they exist, because the interesting part of
# a release here is what the operator has to do, not which commits
# landed. Falls back to a short factual note.
if [ -f "docs/releases/${VERSION}.md" ]; then
NOTES=$(cat "docs/releases/${VERSION}.md")
else
NOTES=$(printf 'Release %s.\n\nSee the repository README for deployment instructions.' "$VERSION")
fi
BODY=$(NOTES="$NOTES" node -e '
const { NOTES, VERSION, IMAGE, DIGEST, REGISTRY } = process.env;
const ref = REGISTRY + "/" + IMAGE + ":" + VERSION;
process.stdout.write(JSON.stringify({
body: NOTES +
"\n\n---\n\n" +
"**Image:** `" + ref + "`\n\n" +
"**Digest:** `" + DIGEST + "`\n\n" +
"```\n" + ref + "\n```\n",
name: "Discord Authenticator " + VERSION,
tag_name: "v" + VERSION,
draft: false,
prerelease: false,
}));')
# A re-run of the same tag updates the release in place rather than
# failing on the duplicate.
curl -s -H "Authorization: token ${GITEA_TOKEN}" \
"${API}/repos/${REPO}/releases/tags/v${VERSION}" > /tmp/release.json || echo '{}' > /tmp/release.json
EXISTING=$(node -e '
const fs = require("fs");
try {
const r = JSON.parse(fs.readFileSync("/tmp/release.json", "utf8"));
process.stdout.write(r && r.id ? String(r.id) : "");
} catch { process.stdout.write(""); }
')
if [ -n "$EXISTING" ]; then
curl -sf -X PATCH -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" -d "$BODY" \
"${API}/repos/${REPO}/releases/${EXISTING}" > /dev/null
echo "updated release v${VERSION}"
else
curl -sf -X POST -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" -d "$BODY" \
"${API}/repos/${REPO}/releases" > /dev/null
echo "created release v${VERSION}"
fi
+89
View File
@@ -0,0 +1,89 @@
## The vault can no longer be destroyed by a crash
This release fixes the failure that could leave the container in a restart loop
with:
```
Error: could not load the vault
Caused by:
/var/lib/discord-authenticator/vault.json exists but could not be parsed
(EOF while parsing a value at line 1 column 0). Refusing to start with an
empty vault — move the file aside to start fresh.
```
### What was wrong
The vault was written as `fs::write(tmp)` → `rename(tmp, path)`. The rename is
atomic, so a reader never saw a half-written file — but **atomicity is not
durability**. Nothing fsynced the new bytes before the rename, so on ext4
(`data=ordered`, the default) a crash or hard power-off could commit the rename
while the data it published never landed. The result is a file that exists and
is zero bytes long, which is exactly what the message above describes.
Two things made it far more likely. The write happened on **every** push tick,
so a ten-account vault was rewriting all of its secrets thousands of times a
day, and there was no backup to fall back on — one damaged file meant a crash
loop, with the web UI that could have fixed it locked behind the service that
would not start.
### What changed
**Durable writes.** `store_io::write_atomic` fsyncs the new bytes, renames, then
fsyncs the directory. The rename now cannot reach the journal ahead of the data
it publishes.
**A backup, and recovery from it.** Every write refreshes `vault.json.bak`. The
backup is taken *after* the new file is safely in place, so the two always hold
the same accounts — backing up first would leave the backup a full change
behind and quietly lose your most recent edit. At startup, an unparseable
`vault.json` now falls back to that backup, logs one `ERROR` explaining what
happened, moves the damaged file to `vault.json.corrupt-<timestamp>` instead of
deleting it, and writes the recovered accounts back so the next boot is
ordinary. With no usable backup it still refuses to start, exactly as before.
**The secrets are no longer rewritten on every code rotation.** Discord
bookkeeping — message ids, last-sent codes, last errors — moved to its own
`push-state.json`, batched and flushed at most every five seconds, with a final
flush on clean shutdown. It is a cache by construction: losing or corrupting it
costs one reposted Discord tile, never a TOTP secret, so it is never a reason to
refuse to boot.
### Upgrading from v0.1.2
Nothing to do. The vault format is unchanged and no accounts need re-importing.
On Unraid, either wait for the Docker page to notice `:latest`, or force it:
```bash
docker pull gitea.milesward.dev/mward4/discord-authenticator:0.1.3
```
then **Force Update** on the container. Pin the image to `0.1.3` rather than
`:latest` in the Unraid template if you would rather updates be deliberate.
Your data directory gains two files: `vault.json.bak` (a copy of the vault —
include it in backups alongside `vault.json`) and `push-state.json` (safe to
delete at any time; the tiles are simply reposted).
### If you already hit this
If your `vault.json` is currently zero bytes, check for a `.bak`:
```bash
cd /mnt/user/appdata/discord-authenticator
ls -la vault* && head -c 100 vault.json.bak
```
`v0.1.3` recovers automatically when that backup parses — just update the image
and start it. If there is no usable backup, the bytes are gone and the accounts
must be re-added from your own `otpauth://` links or a Google Authenticator
export. **Keep `vault.key`**: without it, nothing in an old vault file can be
decrypted even if the bytes turn up later.
### Verifying the fix
143 tests cover this, including recovery from a zero-length vault, retention of
the damaged file, write-back on recovery, refusal when both files are bad,
reload of tile ids across a restart, batching, and an assertion that a code
rotation leaves `vault.json` byte-identical.