Compare commits

..
4 Commits
Author SHA1 Message Date
Miles Ward e80a66b0da v0.8.2: fix iPXE menu item selection (issue #2)
In 0.8.1 on x86_64 UEFI (snponly.efi) the boot menu drew and the keyboard
worked, but every item except the first one bounced straight back to the
menu with the default Queued Deployment entry highlighted; the `x` Exit
item did the same. Root cause was the generated dispatch lines:

  iseq ${target} linux && chain .../boot/_linux_menu.ipxe || goto menu

iPXE evaluates `&&`/`||` strictly left to right (src/core/exec.c
`system()`), so for `A && B || C` the `goto menu` runs whenever A fails --
for every item except the first. Only `local` ever reached its chain line,
which is exactly the reported symptom. The same shape made `x` useless,
and `goto menu` redraws with `--default queue`. The bug report's Ctrl-B
test (chaining an entry script by hand) booted fine, confirming the
scripts and HTTP/TFTP paths were healthy.

Fix, all in crates/http-api/src/ipxe_script.rs:

- Dispatch with `goto ${target} || goto menu` to `:label` blocks (the
  canonical iPXE menu pattern) in the top menu, both installer submenus,
  the tools menu and the utilities menu, so a non-matching item can never
  fall through into a stray `goto menu`.
- Menu-to-menu transitions use `chain --replace`, so the previous menu
  image is replaced: `exit` from any submenu reaches the firmware and
  repeated Back/forward navigation no longer nests script images.
- Boot-entry scripts now recover to the menu themselves (kernel/initrd
  fetches `|| goto failed`, `:failed` chain --replace boot.ipxe), since
  with --replace the calling menu can no longer catch their errors.
- Arch-label mapping: three `|| echo` probes instead of one chain ending
  in `|| true` (`true` is not an iPXE command; non-allowlisted pairs such
  as i386+efi aborted the script before the menu was drawn).
- Utilities menu Back really returns to Tools (it used to redraw
  Utilities); `prompt --timeout` expiry no longer aborts NIC info;
  password prompt tests emptiness with `isset` instead of a literal `""`
  token (iPXE has no quoting, so an empty Enter skipped the main-menu
  path); queue entry view falls back to the main menu if the queue
  endpoint is unreachable.

Tests: new `menu_dispatch_tests` interpreter that mirrors iPXE's
`system()`/`process_script()` (whitespace tokenisation, left-associative
`&&`/`||`, goto labels, abort on nonzero line) and asserts each top-menu
and submenu item reaches its expected chain URL (the old lines would
fail these), plus exit/reboot/firmware, unknown-architecture, password
prompt, shell and queue flows. Text assertions, the insta snapshot and the
tools-menu wiring assertion in tests/full_flow.rs updated to match.
`cargo fmt --check` clean, no new clippy warnings, `cargo test
--workspace` green (all crates).
2026-10-18 06:36:23 -04:00
DSH Agent eef317fb3a dependency-check: run in an image that has both cargo and Node
The first run died on `exec: node: not found` — rust:1.95-bookworm has no Node,
and the report needs cargo to resolve the workspace graph alongside Node to read
the registries. .gitea/scripts/ci.Dockerfile layers the two official images
together, so the parsing exists once rather than once per language.
2026-09-20 02:54:22 -04:00
DSH Agent 16e535212f Add a six-hourly dependency check
Reports drifted crates, the Rust pin and the Docker base images in a single
issue titled "Dependency updates", rewritten in place rather than commented on
and closed automatically when nothing is out of date.

Nothing is upgraded automatically: this reports, a human decides. The report
separates what `cargo update` covers from what crosses a breaking boundary —
including the pre-1.0 rule, where `0.5 -> 0.6` is as breaking as `1.5 -> 2.0`.

The scanner is a standalone script under .gitea/scripts so the same file serves
the sibling Rust repos; the workflow reads the Rust version from this repo's own
Dockerfile rather than hard-coding one, so the check runs on the compiler the
image actually ships with.

Cargo is deliberately run online here: this project resolves a crate from a
private registry, and `--offline` would turn that into no report at all.

Requires a REGISTRY_TOKEN secret.
2026-09-20 02:49:35 -04:00
Miles WardandClaude Opus 4.8 5f98e6e03f v0.8.1: add ISO by URL, zero-touch admin bootstrap
Ease-of-use pass inspired by Bootimus (Dnsmasq-PXE is a manual dnsmasq
setup guide — nothing to adopt; OpenPXE already replaces that stack).

Add ISO by URL:
- New http-api `fetch` module: a small FetchJobs registry + a background
  streaming download (reqwest) that pipes a remote .iso through the same
  UploadHandle + introspection path as an upload, so a URL-fetched image
  classifies and gains boot entries identically. Progress is polled by the
  Storage view and rendered as rows, mirroring uploads.
- Routes POST/GET/DELETE /api/isos/fetch. http/https only; .iso-only
  filename derived from Content-Disposition / URL basename with path
  traversal stripped; 16 GiB cap; cancel; credential-stripped URL display.
  Operator-gated, no boot-time outbound — offline boot is untouched.
- Storage upload card gains an "Or add by URL" field with progress + cancel.

Zero-touch admin bootstrap:
- OPENPXE_ADMIN_USERNAME + OPENPXE_ADMIN_PASSWORD (or _PASSWORD_FILE for
  Docker/K8s secrets) auto-create the admin on first run, so a fresh
  container is usable with no setup wizard. Seeds the first run only — a
  lingering env var can't reset a rotated password.

Tests: URL parse / filename / Content-Disposition unit tests + a wiremock
end-to-end fetch-into-store integration test. clippy/fmt/node clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-16 15:56:27 -04:00
15 changed files with 2036 additions and 94 deletions

No files matched your search

+21
View File
@@ -0,0 +1,21 @@
# CI-only image for the dependency check.
#
# The report is a Node script because it has to compare semantic versions and
# read four different HTTP APIs, and the official `rust` image ships no Node —
# while the `node` image ships no cargo, which `cargo metadata` needs to resolve
# the workspace graph. Rather than maintain the parsing twice in two languages,
# this layers the two together.
#
# Node is copied from the official image rather than installed from a
# third-party apt repository: no extra signing key, no extra network
# dependency at build time, and the same binaries everyone else runs.
ARG RUST_VERSION=1.95
ARG NODE_VERSION=22-bookworm-slim
FROM node:${NODE_VERSION} AS node
FROM rust:${RUST_VERSION}-bookworm
COPY --from=node /usr/local/bin/node /usr/local/bin/node
COPY --from=node /usr/local/lib/node_modules /usr/local/lib/node_modules
RUN ln -sf /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm \
&& node --version
+516
View File
@@ -0,0 +1,516 @@
//! Report every dependency that has a newer release than the one in use.
//!
//! Runs from `.gitea/workflows/dependency-check.yml` on a six-hour schedule.
//! Prints a Markdown report on stdout and exits 0 whether or not anything is
//! out of date — "up to date" is a normal outcome, not a failure. The workflow
//! decides what to do with the report; this file only gathers facts.
//!
//! Three things are checked, because all three go stale independently and none
//! of them is visible from the others:
//!
//! * **Crates** — every locked dependency in `Cargo.lock` against crates.io.
//! * **The Rust pin** — `rust-toolchain.toml`'s channel against the latest
//! stable release, and the Dockerfile's `RUST_VERSION` against both, since
//! those two drifting apart is how a local build and a shipped image end up
//! on different compilers.
//! * **Base images** — the Dockerfile's `FROM` tags against their upstream
//! registries.
//!
//! Network only: no crates are downloaded, nothing is built, and the working
//! tree is never modified.
import { readFileSync, existsSync } from "node:fs";
import { execFileSync } from "node:child_process";
const UA = "dependency-check (+https://gitea.milesward.dev/mward4)";
// ── version comparison ──────────────────────────────────────────────────────
/// Parse `1.2.3`, `1.2`, `1.2.3-alpha.1`, and `=1.2.3` into a comparable
/// tuple. Returns null for anything that isn't a version we understand, so an
/// unparseable requirement is reported as "unknown" rather than silently
/// sorting to the top.
function parseVersion(text) {
const m = /^[=~^]?\s*v?(\d+)(?:\.(\d+))?(?:\.(\d+))?(?:[-+](.*))?$/.exec(text.trim());
if (!m) return null;
return {
major: Number(m[1]),
minor: Number(m[2] ?? 0),
patch: Number(m[3] ?? 0),
pre: m[4] ?? "",
};
}
/// Order two parsed versions. A pre-release sorts below its release, so
/// `2.0.0-rc.1` never displaces a stable `1.9.0` as "the newest".
function compare(a, b) {
for (const key of ["major", "minor", "patch"]) {
if (a[key] !== b[key]) return a[key] < b[key] ? -1 : 1;
}
if (a.pre === b.pre) return 0;
if (!a.pre) return 1;
if (!b.pre) return -1;
return a.pre < b.pre ? -1 : 1;
}
/// How a version difference is allowed to be taken.
///
/// The pre-1.0 rule is the one that matters here: semver treats `0.x.y` as
/// unstable, so `0.5 → 0.6` breaks exactly as hard as `1.5 → 2.0`. Getting this
/// wrong would label half the RustCrypto family "minor or patch" when in fact
/// `cargo update` will refuse every one of them.
function bumpKind(from, to) {
if (to.major !== from.major) return "breaking";
if (from.major === 0 && to.minor !== from.minor) return "breaking";
return "compatible";
}
/// The newest release reached without crossing a breaking boundary — what
/// `cargo update` will move to on its own.
function bestCompatible(from, versions) {
let best = null;
for (const raw of versions) {
const v = parseVersion(raw);
// `bumpKind` is checked before comparing against `from`: `0.14.7` is not
// semver-compatible with `1.0.0` even though it is numerically lower, and
// the naive `compare > 0` test would let `0.14 → 1.4` through as routine.
if (!v || bumpKind(from, v) !== "compatible" || compare(v, from) <= 0) continue;
if (!best || compare(v, best) > 0) best = v;
}
return best;
}
/// The newest release of any kind, which for a breaking bump is what a human
/// would have to adopt.
function bestOverall(from, versions) {
let best = null;
for (const raw of versions) {
const v = parseVersion(raw);
if (!v || compare(v, from) <= 0) continue;
if (!best || compare(v, best) > 0) best = v;
}
return best;
}
const fmt = (v) => (v ? `${v.major}.${v.minor}.${v.patch}${v.pre ? `-${v.pre}` : ""}` : "?");
// ── inputs ──────────────────────────────────────────────────────────────────
/// Resolve the dependency graph with cargo itself.
///
/// `--locked` rather than a hand-rolled Cargo.lock parse: it fails instead of
/// silently rewriting the lockfile, which matters because this runs on a
/// schedule and must never mutate the repository it is reporting on. Cargo.toml
/// is still read directly below for the declared requirement, since metadata
/// reports the resolved version rather than the constraint.
function cargoMetadata() {
let raw;
try {
raw = execFileSync("cargo", ["metadata", "--format-version", "1", "--locked"], {
encoding: "utf8",
maxBuffer: 64 * 1024 * 1024,
stdio: ["ignore", "pipe", "pipe"],
});
} catch (e) {
// A cargo failure here means no report at all. Say why in one line rather
// than letting a raw stack trace stand in for an explanation.
const detail = (e.stderr || e.message || "").toString().trim().split("\n")[0];
throw new Error(`cargo metadata failed: ${detail}`);
}
return JSON.parse(raw);
}
function declaredRequirements() {
// Handles the `dep = "1.2"` and `dep = { version = "1.2", ... }` forms and
// nothing else, which is every dependency this workspace declares.
const text = readFileSync("Cargo.toml", "utf8");
const out = new Map();
let section = "";
for (const line of text.split("\n")) {
const header = /^\[([^\]]+)\]/.exec(line);
if (header) {
section = header[1];
continue;
}
if (!/dependencies/.test(section)) continue;
const m = /^\s*([A-Za-z0-9_-]+)\s*=\s*(.+?)\s*$/.exec(line);
if (!m) continue;
const [, name, value] = m;
const quoted = /^"([^"]+)"/.exec(value);
if (quoted) out.set(name, quoted[1]);
else {
const v = /version\s*=\s*"([^"]+)"/.exec(value);
if (v) out.set(name, v[1]);
}
}
return out;
}
// ── external lookups ────────────────────────────────────────────────────────
/// The latest stable Rust, from the release channel manifest.
///
/// Read from the sparse index rather than the crates.io API. The API is rate
/// limited and answers 429 under a burst of ~130 lookups, which silently
/// turned most of this report into "could not check" on the first run; the
/// index is the same data, one request per crate, no limit and no user agent
/// requirement.
async function publishedVersions(name) {
// The index shards by name length and case: 1-char names live under "1",
// 2-char under "2", 3-char under "3/<first letter>", and everything else
// under the first two characters. Names are lowercased on the way in.
const lower = name.toLowerCase();
const prefix =
lower.length === 1
? `1/${lower}`
: lower.length === 2
? `2/${lower}`
: lower.length === 3
? `3/${lower[0]}/${lower}`
: `${lower.slice(0, 2)}/${lower.slice(2, 4)}/${lower}`;
try {
const res = await fetch(`https://index.crates.io/${prefix}`, {
headers: { "user-agent": UA },
});
// The index answers 404 for a crate that does not exist and 410 for one
// that was removed; both mean "nothing to compare against".
if (!res.ok) return null;
return (await res.text())
.split("\n")
.filter(Boolean)
.map((line) => JSON.parse(line))
.filter((entry) => !entry.yanked && !entry.vers.includes("-"))
.map((entry) => entry.vers);
} catch {
return null;
}
}
/// The latest stable Rust, from the release channel manifest.
async function latestRust() {
try {
const res = await fetch("https://static.rust-lang.org/dist/channel-rust-stable.toml", {
headers: { "user-agent": UA },
});
if (!res.ok) return null;
const m = /\[pkg\.rust\]\nversion = "([^"]+)"/.exec(await res.text());
// The manifest value carries the commit: "1.98.1 (48a229cea 2026-09-01)".
// Only the number is comparable to the Dockerfile's RUST_VERSION.
return m ? m[1].split(/\s+/)[0] : null;
} catch {
return null;
}
}
/// Tags for a Docker Hub repository, or null if it can't be read.
async function dockerTags(repo) {
const [ns, name] = repo.includes("/") ? repo.split("/") : ["library", repo];
try {
const res = await fetch(`https://hub.docker.com/v2/repositories/${ns}/${name}/tags?page_size=100`, {
headers: { "user-agent": UA },
});
if (!res.ok) return null;
const data = await res.json();
return data.results.map((t) => t.name);
} catch {
return null;
}
}
// ── checks ──────────────────────────────────────────────────────────────────
async function checkCrates() {
const meta = cargoMetadata();
const requirements = declaredRequirements();
// One row per crate actually compiled into the workspace. Dev-dependencies
// and build-dependencies are real dependencies too, so they are included —
// a stale build script is still a stale build script.
const seen = new Map();
for (const pkg of meta.packages) {
if (meta.workspace_members.includes(pkg.id)) continue;
seen.set(pkg.name, { from: pkg.version, kind: pkg.source ? "registry" : "path" });
}
const rows = [];
for (const [name, info] of [...seen].sort((a, b) => a[0].localeCompare(b[0]))) {
if (info.kind === "path") continue;
// Direct means "named in Cargo.toml", which is the only set the author can
// act on. The rest move when whatever pulls them in moves, so they are
// reported but never counted as work.
const direct = requirements.has(name);
const current = parseVersion(info.from);
const versions = await publishedVersions(name);
if (!current || !versions) {
rows.push({ name, from: info.from, to: null, kind: "unknown", direct, note: "could not check" });
continue;
}
const compatible = bestCompatible(current, versions);
const overall = bestOverall(current, versions);
if (!overall) continue; // already newest
// `compatible` is what `cargo update` will do unaided. If the newest
// release is further out than that, the gap is a breaking change someone
// has to adopt deliberately.
const kind = compatible ? "compatible" : bumpKind(current, overall);
rows.push({
name,
from: info.from,
to: fmt(overall),
kind,
direct,
note: kind === "compatible" ? "`cargo update`" : "**breaking — needs review**",
});
}
return rows;
}
function rustPin() {
const toolchain = existsSync("rust-toolchain.toml")
? readFileSync("rust-toolchain.toml", "utf8")
: "";
const channel = /channel\s*=\s*"([^"]+)"/.exec(toolchain)?.[1] ?? null;
// The image bakes its own toolchain, so this is the number that decides what
// actually compiles the release binary.
const dockerfile = existsSync("deploy/docker/Dockerfile")
? readFileSync("deploy/docker/Dockerfile", "utf8")
: existsSync("Dockerfile")
? readFileSync("Dockerfile", "utf8")
: "";
const image = /ARG\s+RUST_VERSION=([0-9.]+)/.exec(dockerfile)?.[1] ?? null;
// The number `cargo` enforces as the minimum supported toolchain.
const manifest = readFileSync("Cargo.toml", "utf8");
const msrv = /rust-version\s*=\s*"([^"]+)"/.exec(manifest)?.[1] ?? null;
return { channel, image, msrv };
}
/// Every `FROM` in the Dockerfile, with the `ARG` substitution applied for the
/// one argument this build actually parameterises.
function baseImages() {
const path = existsSync("deploy/docker/Dockerfile")
? "deploy/docker/Dockerfile"
: existsSync("Dockerfile")
? "Dockerfile"
: null;
if (!path) return [];
const text = readFileSync(path, "utf8");
const args = new Map(
[...text.matchAll(/^ARG\s+([A-Za-z0-9_]+)=(\S+)/gm)].map((m) => [m[1], m[2]]),
);
const images = [];
for (const m of text.matchAll(/^FROM\s+(?:--platform=\S+\s+)?(\S+)/gm)) {
let ref = m[1];
// `node:${NODE_VERSION}-slim` and friends.
ref = ref.replace(/\$\{?([A-Za-z0-9_]+)\}?/g, (whole, name) => args.get(name) ?? whole);
if (ref.includes("$")) continue; // still unresolved; skip rather than guess
const [repo, tag = "latest"] = ref.split(":");
images.push({ repo, tag });
}
return images;
}
async function checkRustToolchain() {
const pin = rustPin();
const latest = await latestRust();
return { ...pin, latest };
}
/// The newest tag that keeps the variant suffix the Dockerfile asked for.
///
/// `rust:1.95-bookworm` and `node:22-bookworm-slim` are not interchangeable
/// with their plain counterparts, so a tag is only a candidate if it ends in
/// exactly the suffix already in use.
function bestTagFor(currentTag, tags) {
const m = /^([\d.]+)(-.*)?$/.exec(currentTag);
if (!m) return null;
const [, , suffix = ""] = m;
let best = null;
for (const tag of tags) {
if (suffix) {
if (!tag.endsWith(suffix)) continue;
} else if (!/^[\d.]+$/.test(tag)) {
continue;
}
const v = parseVersion(suffix ? tag.slice(0, -suffix.length) : tag);
// Date-stamped tags like `alpine:20260805` parse as versions but are edge
// or snapshot builds, not releases. No official image we track is anywhere
// near a three-digit major.
if (v && !suffix && v.major >= 100) continue;
if (v && (!best || compare(v, best) > 0)) best = v;
}
return best;
}
async function checkImages() {
const rows = [];
for (const { repo, tag } of baseImages()) {
const tags = await dockerTags(repo);
if (!tags) {
rows.push({ repo, from: tag, to: null, note: "could not check" });
continue;
}
const parsed = parseVersion(tag);
const best = bestTagFor(tag, tags);
if (!parsed || !best) {
rows.push({ repo, from: tag, to: null, note: "tag is not a version" });
continue;
}
// A base image major bump is a whole new distro release; same rule as a
// crate, so it is reported but flagged rather than treated as routine.
const bump = bumpKind(parsed, best);
rows.push({
repo,
from: tag,
to: fmt(best),
note: bump === "compatible" ? "" : "**breaking — needs review**",
});
}
return rows;
}
// ── report ──────────────────────────────────────────────────────────────────
function table(rows, columns) {
const head = `| ${columns.map((c) => c.title).join(" | ")} |`;
const rule = `| ${columns.map(() => "---").join(" | ")} |`;
const body = rows.map((r) => `| ${columns.map((c) => c.get(r)).join(" | ")} |`);
return [head, rule, ...body].join("\n");
}
async function main() {
const [crates, rust, images] = await Promise.all([checkCrates(), checkRustToolchain(), checkImages()]);
// Split by who can act. Only a crate named in Cargo.toml can be moved by
// hand; everything else follows whichever direct dependency pulls it in, so
// listing those as tasks would be noise that trains the reader to ignore the
// report.
const direct = crates.filter((r) => r.direct);
const transitive = crates.filter((r) => !r.direct);
const routine = direct.filter((r) => r.kind === "compatible");
const breaking = direct.filter((r) => r.kind === "breaking");
const rustOutdated = Boolean(rust.latest && rust.image && rust.image !== rust.latest);
// Transitive crates held back by a version boundary upstream has not crossed.
// Worth knowing, not worth acting on.
const heldBack = transitive.filter((r) => r.kind === "breaking");
const parts = [];
const actionable = routine.length + breaking.length + images.length + (rustOutdated ? 1 : 0);
parts.push(
actionable === 0
? "Everything you can act on is current."
: `${routine.length} routine bump${routine.length === 1 ? "" : "s"}, ` +
`${breaking.length} breaking change${breaking.length === 1 ? "" : "s"}` +
(images.length ? `, ${images.length} base image${images.length === 1 ? "" : "s"}` : "") +
(rustOutdated ? ", and a newer Rust" : "") +
".",
);
if (heldBack.length) {
parts.push(
`_A further ${heldBack.length} transitive crates have newer releases blocked behind a ` +
"breaking boundary upstream has not crossed yet. They are omitted below because " +
"nothing here can move them._",
);
}
if (routine.length) {
parts.push(
"### Direct dependencies — `cargo update` covers these\n\n" +
table(routine, [
{ title: "Crate", get: (r) => `\`${r.name}\`` },
{ title: "In use", get: (r) => `\`${r.from}\`` },
{ title: "Latest", get: (r) => `\`${r.to}\`` },
]),
);
}
if (breaking.length) {
parts.push(
"### Direct dependencies — breaking, a human has to decide\n\n" +
"These cross a version boundary where semver allows anything to change. " +
"`cargo update` will not touch them: `Cargo.toml` needs editing and the " +
"changelog reading.\n\n" +
table(breaking, [
{ title: "Crate", get: (r) => `\`${r.name}\`` },
{ title: "In use", get: (r) => `\`${r.from}\`` },
{ title: "Latest", get: (r) => `\`${r.to}\`` },
]),
);
}
const rustRows = rust.latest
? [
{
what: "Dockerfile `RUST_VERSION`",
using: rust.image ?? "not pinned",
latest: rust.latest,
note: rustOutdated ? "**the shipped binary is built by an older compiler**" : "current",
},
{
what: "`rust-toolchain.toml` channel",
using: rust.channel ?? "unknown",
latest: rust.latest,
note:
rust.channel === "stable"
? "tracks stable, so it is already current"
: "pinned to a fixed release",
},
]
: [
{
what: "latest stable Rust",
using: rust.image ?? "unknown",
latest: "—",
note: "could not reach static.rust-lang.org",
},
];
parts.push(
"### Rust\n\n" +
table(rustRows, [
{ title: "What", get: (r) => r.what },
{ title: "Declared", get: (r) => `\`${r.using}\`` },
{ title: "Latest", get: (r) => `\`${r.latest}\`` },
{ title: "Note", get: (r) => r.note },
]) +
(rust.msrv ? `\n\nMinimum supported Rust version: \`${rust.msrv}\`.` : ""),
);
if (images.length) {
parts.push(
"### Base images\n\n" +
table(images, [
{ title: "Image", get: (r) => `\`${r.repo}\`` },
{ title: "In use", get: (r) => `\`${r.from}\`` },
{ title: "Latest", get: (r) => `\`${r.to}\`` },
{ title: "Note", get: (r) => r.note },
]),
);
}
parts.push("_Checked every 6 hours by `.gitea/workflows/dependency-check.yml`._");
console.log(parts.join("\n\n"));
// Machine-readable footer, so the workflow never has to parse prose.
console.error(
`::dep-check:: routine=${routine.length} breaking=${breaking.length} ` +
`images=${images.length} rust=${rustOutdated ? 1 : 0}`,
);
}
try {
await main();
} catch (e) {
// Print the reason and exit non-zero. The workflow treats a missing report as
// a broken check rather than a clean bill of health, so this keeps the
// tracking issue open instead of closing it on no evidence.
console.error(`dependency check failed: ${e.message}`);
process.exit(1);
}
+184
View File
@@ -0,0 +1,184 @@
name: Check for dependency updates
# Runs every six hours and reports anything out of date in one place: the
# crates in Cargo.lock, the Rust pin in rust-toolchain.toml and the Dockerfile,
# and the Dockerfile's base images.
#
# Everything lands in ONE issue titled "Dependency updates", edited in place
# rather than commented on, so there is a single current answer instead of a
# pile of history. When nothing is out of date the issue is closed — an empty
# issue left open is a thing people stop reading, which defeats the point.
#
# Nothing is upgraded automatically. `cargo update` on a schedule would land
# unreviewed code in a service that holds the operator's second factor, and a
# breaking bump needs a human regardless.
on:
schedule:
# Every six hours, off the hour so this doesn't compete with an image
# rebuild for a runner.
- cron: '23 */6 * * *'
workflow_dispatch:
# One run at a time. A manual dispatch during a scheduled run would otherwise
# race it and could open the issue twice.
concurrency:
group: dependency-check
cancel-in-progress: false
permissions:
contents: read
issues: write
jobs:
check:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v4
# The toolchain comes from the repo rather than being hard-coded here, so
# the check runs on the same compiler the image ships with and this file
# can be copied between repos unchanged.
- name: Resolve the pinned Rust version
id: rust
run: |
set -eu
FILE=deploy/docker/Dockerfile
[ -f "$FILE" ] || FILE=Dockerfile
VERSION=$(grep -E '^ARG RUST_VERSION=' "$FILE" | head -1 | cut -d= -f2)
[ -n "$VERSION" ] || { echo "::error::no ARG RUST_VERSION in $FILE"; exit 1; }
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "checking with rust ${VERSION}"
# The workspace is streamed into the container over stdin rather than
# bind-mounted, so this works whether the runner shares a filesystem with
# the daemon or not.
#
# The image is rust + node, built from .gitea/scripts/ci.Dockerfile: the
# report needs cargo to resolve the workspace graph and Node to read the
# registries, and neither official image has both.
#
# RUSTUP_TOOLCHAIN overrides rust-toolchain.toml: the image's own
# toolchain is enough to read Cargo.lock, and honouring the file would
# make every run fetch a second compiler for no benefit.
#
# Cargo is deliberately *not* run offline: at least one project here
# depends on a crate published to a private registry, and `--offline`
# turns that into a failed resolution and therefore no report at all.
- name: Generate report
run: |
set -eu
VERSION="${{ steps.rust.outputs.version }}"
docker build \
--build-arg "RUST_VERSION=${VERSION}" \
--tag "dep-check:${VERSION}" \
--file .gitea/scripts/ci.Dockerfile \
.gitea/scripts > /dev/null
tar cf - --exclude=./.git --exclude=./target . | docker run --rm -i \
-e RUSTUP_TOOLCHAIN="${VERSION}" \
-e CARGO_TERM_COLOR=never \
"dep-check:${VERSION}" \
sh -ec 'mkdir -p /work && cd /work && tar xf - && exec node .gitea/scripts/dependency-report.mjs' \
> /tmp/report.md 2> /tmp/report.err || {
echo "::error::the dependency report failed to generate"
cat /tmp/report.err
exit 1
}
cat /tmp/report.md
grep -E '::dep-check::' /tmp/report.err || true
# An empty report is a broken check, not a clean bill of health.
# Failing here keeps the tracking issue open rather than closing it on
# the strength of a file that never got written.
test -s /tmp/report.md
grep -oE 'routine=[0-9]+ breaking=[0-9]+ images=[0-9]+ rust=[0-9]+' /tmp/report.err \
> /tmp/counts
cat /tmp/counts
- name: Open, update or close the tracking issue
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
REPO: ${{ github.repository }}
run: |
set -eu
API="https://gitea.milesward.dev/api/v1"
node -e '
const fs = require("fs");
const report = fs.readFileSync("/tmp/report.md", "utf8").trim();
const counts = fs.readFileSync("/tmp/counts", "utf8").trim();
const [routine, breaking, images, rust] = counts.match(/[0-9]+/g).map(Number);
const actionable = routine + breaking + images + rust;
const body =
(actionable === 0
? "Everything tracked is current. Closing this until something changes.\n\n"
: "") +
report +
"\n\n---\n\n" +
"Rewritten by `.gitea/workflows/dependency-check.yml` every 6 hours, so this is " +
"always the latest report rather than a pile of comments.";
fs.writeFileSync("/tmp/state.json", JSON.stringify({ actionable, body }));
'
ACTIONABLE=$(node -e 'process.stdout.write(String(require("/tmp/state.json").actionable))')
echo "actionable=$ACTIONABLE"
# Existing issue by title, any state: a previously closed one is
# exactly the one to update and reopen, and searching all states is
# also what stops a duplicate being created after a close.
#
# The `|| true` matters: `set -e` plus `pipefail` would abort the step
# on a transient API error, and creating a second issue is a far worse
# outcome than missing one run.
NUMBER=$(curl -sf -H "Authorization: token ${GITEA_TOKEN}" \
"${API}/repos/${REPO}/issues?state=all&type=issues&limit=100" \
| node -e '
let s = "";
process.stdin.on("data", d => s += d).on("end", () => {
try {
const issues = JSON.parse(s);
const hit = issues.find(i => i.title === "Dependency updates");
process.stdout.write(hit ? String(hit.number) : "");
} catch { process.stdout.write(""); }
});
' || true)
BODY=$(node -e 'process.stdout.write(JSON.stringify({ body: require("/tmp/state.json").body }))')
if [ -z "$NUMBER" ]; then
if [ "$ACTIONABLE" = "0" ]; then
echo "everything current and no issue open — nothing to do"
exit 0
fi
curl -sf -X POST -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
-d "$(node -e 'process.stdout.write(JSON.stringify({ title: "Dependency updates", body: require("/tmp/state.json").body }))')" \
"${API}/repos/${REPO}/issues" > /dev/null
echo "opened the tracking issue"
exit 0
fi
curl -sf -X PATCH -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
-d "$BODY" "${API}/repos/${REPO}/issues/${NUMBER}" > /dev/null
echo "updated issue #${NUMBER}"
# Reopen when there is work, close when there isn't. Done as a second
# call because the state is not part of the body above.
if [ "$ACTIONABLE" = "0" ]; then WANT=closed; else WANT=open; fi
STATE=$(curl -sf -H "Authorization: token ${GITEA_TOKEN}" \
"${API}/repos/${REPO}/issues/${NUMBER}" \
| node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>process.stdout.write(JSON.parse(s).state))')
if [ "$STATE" != "$WANT" ]; then
curl -sf -X PATCH -H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"state\":\"${WANT}\"}" "${API}/repos/${REPO}/issues/${NUMBER}" > /dev/null
echo "issue #${NUMBER}: ${STATE} -> ${WANT}"
fi
Generated
+8 -8
View File
@@ -2758,7 +2758,7 @@ checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
[[package]]
name = "openpxe"
version = "0.8.0"
version = "0.8.2"
dependencies = [
"anyhow",
"axum",
@@ -2778,7 +2778,7 @@ dependencies = [
[[package]]
name = "openpxe-core"
version = "0.8.0"
version = "0.8.2"
dependencies = [
"anyhow",
"base64",
@@ -2803,7 +2803,7 @@ dependencies = [
[[package]]
name = "openpxe-dhcp-proxy"
version = "0.8.0"
version = "0.8.2"
dependencies = [
"anyhow",
"dhcproto",
@@ -2818,7 +2818,7 @@ dependencies = [
[[package]]
name = "openpxe-http-api"
version = "0.8.0"
version = "0.8.2"
dependencies = [
"anyhow",
"axum",
@@ -2852,7 +2852,7 @@ dependencies = [
[[package]]
name = "openpxe-ipxe-assets"
version = "0.8.0"
version = "0.8.2"
dependencies = [
"openpxe-core",
"rust-embed",
@@ -2861,7 +2861,7 @@ dependencies = [
[[package]]
name = "openpxe-iso-store"
version = "0.8.0"
version = "0.8.2"
dependencies = [
"anyhow",
"bcrypt",
@@ -2887,7 +2887,7 @@ dependencies = [
[[package]]
name = "openpxe-tftp"
version = "0.8.0"
version = "0.8.2"
dependencies = [
"anyhow",
"openpxe-core",
@@ -2899,7 +2899,7 @@ dependencies = [
[[package]]
name = "openpxe-webui"
version = "0.8.0"
version = "0.8.2"
[[package]]
name = "p256"
+1 -1
View File
@@ -12,7 +12,7 @@ members = [
]
[workspace.package]
version = "0.8.0"
version = "0.8.2"
edition = "2021"
rust-version = "1.95"
license = "MIT OR Apache-2.0"
+9 -1
View File
@@ -65,7 +65,10 @@ OpenPXE collapses that whole stack into **one statically-linked binary in one co
hierarchy — generated fresh on every request from current settings.
#### ISO management & remote libraries
- **Drag-and-drop chunked uploads** that don't 502 on multi-GB images.
- **Drag-and-drop chunked uploads** that don't 502 on multi-GB images — drop several at once
and they upload concurrently.
- **Add by URL** — paste an ISO link and the server streams it straight into storage and
auto-detects it, with progress; no download-then-reupload.
- **Automatic introspection** — detects the distro family and generates the right
kernel+initrd or Windows `wimboot` chain. No manual config.
- **Remote ISO libraries, streamed on demand** (no local cache) over **SMB, NFS, or SFTP** —
@@ -88,6 +91,9 @@ OpenPXE collapses that whole stack into **one statically-linked binary in one co
- **Prometheus `/metrics`**, a built-in operator **terminal**, live tracing log, and
`/healthz` · `/readyz` probes.
- **Layered config** — defaults → TOML file → `OPENPXE_*` env, in that order.
- **Zero-touch first run** — set `OPENPXE_ADMIN_USERNAME` + `OPENPXE_ADMIN_PASSWORD` and a
fresh container comes up with the admin already created, no setup wizard. Automate the rest
from scripts/Postman with a per-install **API key** (`x-api-key`), shown in Settings → Advanced.
## Built in Rust
@@ -236,6 +242,8 @@ All settings have defaults and layer **defaults → TOML (`--config` / `OPENPXE_
| `OPENPXE_ISO_DIR` | `/var/lib/openpxe/isos` | Uploaded ISOs |
| `OPENPXE_WORK_DIR` | `/var/lib/openpxe/work` | Scratch, settings, share + branding state |
| `OPENPXE_LOG` | `info,openpxe=info` | `tracing` filter |
| `OPENPXE_ADMIN_USERNAME` | — | First-run only: with `OPENPXE_ADMIN_PASSWORD`, auto-creates the admin so no setup wizard is needed. Ignored once an admin exists. |
| `OPENPXE_ADMIN_PASSWORD` | — | First-run admin password. Use `OPENPXE_ADMIN_PASSWORD_FILE` to read it from a file (Docker/K8s secret). |
## OpenShift
+17
View File
@@ -94,6 +94,17 @@ pub fn build_router(state: AppState) -> Router {
// JSON API.
.route("/api/isos", get(api_list_isos).post(api_upload_iso))
.route("/api/isos/{id}", delete(api_delete_iso))
// v0.8.1: add ISO by URL — server-side streaming download + progress
// polling. Static `fetch` coexists with `{id}` above (matchit
// prioritizes the literal), same as `/api/queue/join` vs `{entry_id}`.
.route(
"/api/isos/fetch",
get(crate::fetch::api_iso_fetch_list).post(crate::fetch::api_iso_fetch_start),
)
.route(
"/api/isos/fetch/{id}",
delete(crate::fetch::api_iso_fetch_cancel),
)
.route("/api/uploads", post(api_upload_begin))
.route(
"/api/uploads/{upload_id}",
@@ -1857,6 +1868,12 @@ async fn api_docs() -> Json<serde_json::Value> {
"summary": "Clear an ISO's boot password."},
{"method": "PUT", "path": "/api/isos/{id}/category",
"summary": "Set the menu category. Body: { \"category\": \"os\" | \"tools\" }."},
{"method": "POST", "path": "/api/isos/fetch",
"summary": "Add an ISO by URL. Body: { \"url\", \"filename\"? }. The server streams the .iso into storage and introspects it. Returns { fetch_id }."},
{"method": "GET", "path": "/api/isos/fetch",
"summary": "Poll URL-fetch progress — [{ id, filename, url, downloaded, total, state }]. Completed jobs are returned once."},
{"method": "DELETE", "path": "/api/isos/fetch/{id}",
"summary": "Cancel an in-flight URL fetch, or dismiss a finished/failed one."},
],
},
{
+485
View File
@@ -0,0 +1,485 @@
//! Server-side "add ISO by URL" — stream a remote `.iso` straight into the
//! store, reusing the chunked-upload handle + introspection pipeline so a
//! URL-fetched image classifies and gains boot entries exactly like an
//! uploaded one. A small in-memory job registry tracks progress; the web UI
//! polls it and renders rows just like browser uploads.
//!
//! This is operator-initiated and auth-gated (`/api/*`), never runs at boot,
//! and adds no CDN assets — so it doesn't touch OpenPXE's offline-boot
//! guarantee. On an air-gapped network it simply goes unused (upload
//! instead). It's the same class of optional outbound the server already
//! makes for webhooks and the update check.
use crate::state::AppState;
use axum::{
extract::{Path, State},
http::{header, StatusCode},
response::{IntoResponse, Response},
Json,
};
use futures::StreamExt;
use openpxe_core::{Error, Result};
use openpxe_iso_store::IsoStore;
use parking_lot::Mutex;
use serde::{Deserialize, Serialize};
use serde_json::json;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::{Duration, Instant};
use uuid::Uuid;
/// Hard ceiling on a URL-fetched image — matches the HTTP upload body cap
/// (`DefaultBodyLimit` in `app.rs`).
const MAX_ISO_BYTES: u64 = 16 * 1024 * 1024 * 1024;
/// How long a finished-with-error job lingers so the operator can read the
/// failure before it's swept. Successful jobs are read-once (see
/// [`FetchJobs::snapshot`]).
const FAILED_TTL: Duration = Duration::from_mins(10);
#[derive(Clone)]
enum Phase {
Downloading,
Done { iso_id: String },
Failed { error: String },
Canceled,
}
struct Job {
/// Best-known target filename (provisional from the URL, refined once
/// the response headers arrive).
filename: String,
/// Display-safe source URL — any `user:pass@` userinfo is stripped so
/// the UI/logs never echo embedded credentials.
url: String,
downloaded: u64,
/// Total bytes from `Content-Length`, or `0` when the server didn't
/// send one (progress then shows bytes-so-far without a percentage).
total: u64,
phase: Phase,
cancel: bool,
finished_at: Option<Instant>,
}
/// One row in the fetch-progress list the UI polls.
#[derive(Serialize)]
pub struct JobDto {
id: String,
filename: String,
url: String,
downloaded: u64,
total: u64,
/// `downloading` | `done` | `failed` | `canceled`.
state: &'static str,
#[serde(skip_serializing_if = "Option::is_none")]
iso_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
error: Option<String>,
}
/// In-memory registry of in-flight and recently-finished URL fetches.
/// Cheap to clone (Arc-shared); contention is nil (a handful of jobs, brief
/// per-chunk progress bumps).
#[derive(Clone, Default)]
pub struct FetchJobs {
inner: Arc<Mutex<HashMap<String, Arc<Mutex<Job>>>>>,
}
impl FetchJobs {
/// Validate the URL, register a job, and spawn the background download.
/// Returns the job id. Fails fast on a bad/unsupported URL so the POST
/// gets an immediate error instead of a job that dies a moment later.
pub fn start(
&self,
store: IsoStore,
raw_url: &str,
filename_hint: Option<&str>,
) -> Result<String> {
let (url, display) = parse_and_sanitize(raw_url)?;
// Provisional filename for the first render; the task refines it
// from Content-Disposition / the post-redirect URL.
let provisional = filename_hint
.map(sanitize_filename)
.or_else(|| basename(&url))
.unwrap_or_else(|| "download.iso".to_string());
let id = Uuid::new_v4().simple().to_string();
let job = Arc::new(Mutex::new(Job {
filename: provisional,
url: display,
downloaded: 0,
total: 0,
phase: Phase::Downloading,
cancel: false,
finished_at: None,
}));
self.inner.lock().insert(id.clone(), job.clone());
let hint = filename_hint.map(str::to_string);
tokio::spawn(async move {
if let Err(e) = download(&store, &job, url, hint).await {
let mut g = job.lock();
// A cancel flips the phase itself; don't overwrite it.
if !matches!(g.phase, Phase::Canceled) {
g.phase = Phase::Failed {
error: format!("{e}"),
};
}
g.finished_at = Some(Instant::now());
}
});
Ok(id)
}
/// Snapshot every job for the UI, then sweep the terminal ones:
/// `Done`/`Canceled` are **read-once** (removed after this call, so the
/// UI reacts to completion exactly once and never loops on a lingering
/// "done" row), while `Failed` is retained until [`FAILED_TTL`] so the
/// error stays visible.
pub fn snapshot(&self) -> Vec<JobDto> {
let mut g = self.inner.lock();
let mut out = Vec::with_capacity(g.len());
for (id, job) in g.iter() {
let j = job.lock();
let (state, iso_id, error) = match &j.phase {
Phase::Downloading => ("downloading", None, None),
Phase::Done { iso_id } => ("done", Some(iso_id.clone()), None),
Phase::Failed { error } => ("failed", None, Some(error.clone())),
Phase::Canceled => ("canceled", None, None),
};
out.push(JobDto {
id: id.clone(),
filename: j.filename.clone(),
url: j.url.clone(),
downloaded: j.downloaded,
total: j.total,
state,
iso_id,
error,
});
}
let now = Instant::now();
g.retain(|_, job| {
let j = job.lock();
match &j.phase {
Phase::Downloading => true,
Phase::Done { .. } | Phase::Canceled => false, // read-once
Phase::Failed { .. } => j
.finished_at
.is_none_or(|t| now.duration_since(t) < FAILED_TTL),
}
});
out
}
/// Cancel an in-flight download, or dismiss a terminal one. Returns
/// `true` if a job with that id existed.
pub fn cancel(&self, id: &str) -> bool {
let g = self.inner.lock();
let Some(job) = g.get(id) else { return false };
let mut j = job.lock();
if matches!(j.phase, Phase::Downloading) {
j.cancel = true; // the download loop checks this each chunk
} else {
drop(j);
drop(g);
self.inner.lock().remove(id);
}
true
}
}
/// The background download: GET the URL, derive + validate the filename,
/// then stream the body through an `UploadHandle` (which hashes, writes the
/// `.partial`, and on `finish` renames + introspects).
async fn download(
store: &IsoStore,
job: &Arc<Mutex<Job>>,
url: reqwest::Url,
filename_hint: Option<String>,
) -> Result<()> {
let client = reqwest::Client::builder()
.connect_timeout(Duration::from_secs(15))
.user_agent(concat!("OpenPXE/", env!("CARGO_PKG_VERSION")))
.build()
.map_err(|e| Error::Other(e.into()))?;
let resp = client
.get(url)
.send()
.await
.map_err(|e| Error::Invalid(format!("request failed: {e}")))?;
if !resp.status().is_success() {
return Err(Error::Invalid(format!("server returned {}", resp.status())));
}
let cd = resp
.headers()
.get(header::CONTENT_DISPOSITION)
.and_then(|v| v.to_str().ok());
let filename = pick_filename(filename_hint.as_deref(), cd, resp.url())?;
let total = resp.content_length().unwrap_or(0);
if total > MAX_ISO_BYTES {
return Err(Error::Invalid(format!(
"declared size {total} exceeds the {MAX_ISO_BYTES}-byte cap"
)));
}
{
let mut g = job.lock();
g.filename.clone_from(&filename);
g.total = total;
}
let mut handle = store.begin_upload(&filename).await?;
let mut stream = resp.bytes_stream();
let mut received: u64 = 0;
while let Some(item) = stream.next().await {
if job.lock().cancel {
let _ = handle.abort().await;
job.lock().phase = Phase::Canceled;
job.lock().finished_at = Some(Instant::now());
return Ok(());
}
let chunk = item.map_err(|e| Error::Invalid(format!("transfer error: {e}")))?;
received += chunk.len() as u64;
if received > MAX_ISO_BYTES {
let _ = handle.abort().await;
return Err(Error::Invalid(format!(
"download exceeded the {MAX_ISO_BYTES}-byte cap"
)));
}
if let Err(e) = handle.write_chunk(&chunk).await {
let _ = handle.abort().await;
return Err(e);
}
job.lock().downloaded = received;
}
let meta = handle.finish(store).await?;
tracing::info!(
target: "openpxe::http::fetch",
filename = %filename, bytes = received, family = ?meta.introspection.family,
"fetched ISO from URL"
);
let mut g = job.lock();
g.downloaded = received;
g.phase = Phase::Done { iso_id: meta.id };
g.finished_at = Some(Instant::now());
Ok(())
}
/// Parse the URL, require an `http`/`https` scheme (no `file:`/`gopher:`/…),
/// and return it alongside a credential-stripped display form.
fn parse_and_sanitize(raw: &str) -> Result<(reqwest::Url, String)> {
let url = reqwest::Url::parse(raw.trim())
.map_err(|_| Error::Invalid("not a valid URL".to_string()))?;
if !matches!(url.scheme(), "http" | "https") {
return Err(Error::Invalid(
"only http:// and https:// URLs are accepted".to_string(),
));
}
let mut display = url.clone();
let _ = display.set_username("");
let _ = display.set_password(None);
Ok((url, display.to_string()))
}
/// Choose the target filename: explicit hint > `Content-Disposition` >
/// post-redirect URL basename. Must end in `.iso` (case-insensitive).
fn pick_filename(
explicit: Option<&str>,
content_disposition: Option<&str>,
final_url: &reqwest::Url,
) -> Result<String> {
let candidate = explicit
.map(sanitize_filename)
.or_else(|| content_disposition.and_then(filename_from_disposition))
.or_else(|| basename(final_url))
.ok_or_else(|| Error::Invalid("could not determine a filename".to_string()))?;
if !candidate.to_ascii_lowercase().ends_with(".iso") {
return Err(Error::Invalid(format!(
"URL does not point at an .iso (got '{candidate}')"
)));
}
Ok(candidate)
}
/// Last path segment of a URL, percent-decoded and reduced to a bare
/// filename. `None` for a pathless URL.
fn basename(url: &reqwest::Url) -> Option<String> {
let seg = url.path_segments()?.next_back()?;
if seg.is_empty() {
return None;
}
let decoded = percent_decode(seg);
Some(sanitize_filename(&decoded))
}
/// Pull `filename="x.iso"` (or bare `filename=x.iso`) out of a
/// `Content-Disposition` header. RFC 5987 `filename*` is ignored — the
/// common case is enough, and the URL basename is the fallback.
fn filename_from_disposition(cd: &str) -> Option<String> {
let idx = cd.to_ascii_lowercase().find("filename=")?;
let rest = &cd[idx + "filename=".len()..];
let val = rest.trim_start().trim_start_matches('"');
let end = val.find(['"', ';']).unwrap_or(val.len());
let name = val[..end].trim();
if name.is_empty() {
None
} else {
Some(sanitize_filename(name))
}
}
/// Reduce any path-ish string to a safe bare filename: last component only,
/// no `/`, `\`, or NULs. Prevents a crafted `Content-Disposition`/URL from
/// escaping the ISO directory.
fn sanitize_filename(s: &str) -> String {
s.rsplit(['/', '\\'])
.next()
.unwrap_or(s)
.replace('\0', "")
.trim()
.to_string()
}
/// Minimal percent-decoding for a single path segment (enough for `%20`
/// spaces in an ISO name); leaves malformed escapes untouched.
fn percent_decode(s: &str) -> String {
let bytes = s.as_bytes();
let mut out = Vec::with_capacity(bytes.len());
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'%' && i + 2 < bytes.len() {
if let (Some(h), Some(l)) = (hexval(bytes[i + 1]), hexval(bytes[i + 2])) {
out.push(h << 4 | l);
i += 3;
continue;
}
}
out.push(bytes[i]);
i += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
fn hexval(b: u8) -> Option<u8> {
match b {
b'0'..=b'9' => Some(b - b'0'),
b'a'..=b'f' => Some(b - b'a' + 10),
b'A'..=b'F' => Some(b - b'A' + 10),
_ => None,
}
}
// ── Handlers ──────────────────────────────────────────────────────────────
#[derive(Deserialize)]
pub struct FetchBody {
pub url: String,
#[serde(default)]
pub filename: Option<String>,
}
/// `POST /api/isos/fetch` — start a URL download.
pub async fn api_iso_fetch_start(
State(state): State<AppState>,
Json(body): Json<FetchBody>,
) -> Response {
match state
.fetch_jobs
.start(state.iso_store.clone(), &body.url, body.filename.as_deref())
{
Ok(id) => (StatusCode::ACCEPTED, Json(json!({ "fetch_id": id }))).into_response(),
Err(Error::Invalid(msg)) => {
(StatusCode::BAD_REQUEST, Json(json!({ "error": msg }))).into_response()
}
Err(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "error": format!("{e}") })),
)
.into_response(),
}
}
/// `GET /api/isos/fetch` — poll progress. Successful jobs appear once.
pub async fn api_iso_fetch_list(State(state): State<AppState>) -> Response {
(
StatusCode::OK,
Json(json!({ "jobs": state.fetch_jobs.snapshot() })),
)
.into_response()
}
/// `DELETE /api/isos/fetch/{id}` — cancel an in-flight download or dismiss a
/// finished/failed row.
pub async fn api_iso_fetch_cancel(
State(state): State<AppState>,
Path(id): Path<String>,
) -> Response {
if state.fetch_jobs.cancel(&id) {
StatusCode::NO_CONTENT.into_response()
} else {
(
StatusCode::NOT_FOUND,
Json(json!({ "error": "no such fetch job" })),
)
.into_response()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn scheme_is_restricted_and_credentials_are_stripped() {
assert!(parse_and_sanitize("file:///etc/passwd").is_err());
assert!(parse_and_sanitize("gopher://x/1").is_err());
assert!(parse_and_sanitize("not a url").is_err());
let (_, display) = parse_and_sanitize("https://user:[email protected]/a.iso").unwrap();
assert!(
!display.contains("secret"),
"credentials must be stripped: {display}"
);
assert!(display.starts_with("https://example.com/"));
}
#[test]
fn filename_prefers_explicit_then_disposition_then_url() {
let u = reqwest::Url::parse("https://example.com/path/final.iso").unwrap();
// Explicit wins, and path traversal is stripped.
assert_eq!(
pick_filename(Some("../../evil/custom.iso"), None, &u).unwrap(),
"custom.iso"
);
// Content-Disposition next.
assert_eq!(
pick_filename(None, Some(r#"attachment; filename="rescue.iso""#), &u).unwrap(),
"rescue.iso"
);
// URL basename fallback (with percent-decoding).
let sp = reqwest::Url::parse("https://example.com/System%20Rescue.iso").unwrap();
assert_eq!(pick_filename(None, None, &sp).unwrap(), "System Rescue.iso");
// Non-.iso is rejected.
assert!(pick_filename(
None,
None,
&reqwest::Url::parse("https://x/y.tar.gz").unwrap()
)
.is_err());
}
#[test]
fn disposition_parsing_handles_quotes_and_bare() {
assert_eq!(
filename_from_disposition(r#"attachment; filename="a.iso"; size=1"#).as_deref(),
Some("a.iso")
);
assert_eq!(
filename_from_disposition("inline; filename=b.iso").as_deref(),
Some("b.iso")
);
assert_eq!(filename_from_disposition("attachment").as_deref(), None);
}
}
+612 -72
View File
@@ -85,10 +85,25 @@ pub fn render_menu(isos: &[IsoMeta], settings: &Settings, base_url: &str) -> Str
// iPXE evaluates `iseq` lazily, so we only set whichever line
// matches. Anything not on the allowlist falls through to a generic
// `<buildarch> <platform>` display.
//
// v0.8.2: one probe per line, each terminated with `|| echo` (`echo`
// is the harmless rc-0 no-op; there is no `true` command in iPXE).
// The previous single line tried to chain the probes with `||` and
// ended in `|| true`, which is not an iPXE command: on any
// arch/platform pair outside the allowlist (e.g. i386 + efi) the
// command failed and aborted the script before the menu was drawn.
let _ = writeln!(s, "set arch-label ${{buildarch}} ${{platform}}");
let _ = writeln!(
s,
"iseq ${{buildarch}} i386 && iseq ${{platform}} pcbios && set arch-label x86 BIOS || iseq ${{buildarch}} x86_64 && iseq ${{platform}} efi && set arch-label x86_64 UEFI || iseq ${{buildarch}} arm64 && iseq ${{platform}} efi && set arch-label arm64 UEFI || true"
"iseq ${{buildarch}} i386 && iseq ${{platform}} pcbios && set arch-label x86 BIOS || echo"
);
let _ = writeln!(
s,
"iseq ${{buildarch}} x86_64 && iseq ${{platform}} efi && set arch-label x86_64 UEFI || echo"
);
let _ = writeln!(
s,
"iseq ${{buildarch}} arm64 && iseq ${{platform}} efi && set arch-label arm64 UEFI || echo"
);
let _ = writeln!(s, ":menu");
let _ = writeln!(s, "menu OpenPXE - network boot menu");
@@ -151,34 +166,44 @@ pub fn render_menu(isos: &[IsoMeta], settings: &Settings, base_url: &str) -> Str
);
}
// iPXE's `||` is strict about what follows. Each test uses `goto menu`
// as the fallthrough target so the parser never sees a bare `||` with
// trailing whitespace — some iPXE builds reject that.
// v0.8.2: dispatch via `goto <item-value>` labels. This is the
// canonical iPXE menu pattern (ipxe.org examples) and, unlike
// `iseq ... && chain ... || goto menu`, it is not sensitive to the
// left-associative `&&`/`||` evaluation order that broke item
// selection in issue #2: for `A && B || C` iPXE runs C whenever A
// fails, so every item except the first used to skip its chain and
// bounce straight back to `goto menu` — redrawing the menu with the
// default (Queued Deployment) item highlighted, as if Enter did
// nothing.
//
// Menu-to-menu transitions use `chain --replace`: the replacement
// image becomes the top-most image, so `exit` from any submenu
// reaches the firmware and repeated Back/forward navigation does not
// nest script images (iPXE tail-recurses into replacements, so the
// stack stays flat). A failed fetch still returns an error, which
// the trailing `|| goto menu` turns into a menu redraw.
let _ = writeln!(s, "goto ${{target}} || goto menu");
let _ = writeln!(s, ":local");
let _ = writeln!(s, "chain --replace {base}/boot/_local.ipxe || goto menu");
let _ = writeln!(s, ":linux");
let _ = writeln!(
s,
"iseq ${{target}} local && chain {base}/boot/_local.ipxe || goto menu"
"chain --replace {base}/boot/_linux_menu.ipxe || goto menu"
);
let _ = writeln!(s, ":windows");
let _ = writeln!(
s,
"iseq ${{target}} linux && chain {base}/boot/_linux_menu.ipxe || goto menu"
"chain --replace {base}/boot/_windows_menu.ipxe || goto menu"
);
let _ = writeln!(s, ":tools");
let _ = writeln!(
s,
"iseq ${{target}} windows && chain {base}/boot/_windows_menu.ipxe || goto menu"
"chain --replace {base}/boot/_tools_menu.ipxe || goto menu"
);
let _ = writeln!(
s,
"iseq ${{target}} tools && chain {base}/boot/_tools_menu.ipxe || goto menu"
);
let _ = writeln!(
s,
"iseq ${{target}} queue && chain {base}/boot/_queue.ipxe || goto menu"
);
let _ = writeln!(
s,
"iseq ${{target}} exit && exit || goto menu"
);
let _ = writeln!(s, "goto menu");
let _ = writeln!(s, ":queue");
let _ = writeln!(s, "chain --replace {base}/boot/_queue.ipxe || goto menu");
let _ = writeln!(s, ":exit");
let _ = writeln!(s, "exit 0");
s
}
@@ -244,18 +269,21 @@ pub fn render_family_menu(isos: &[IsoMeta], base_url: &str, is_windows: bool) ->
let _ = writeln!(s, "item --gap");
let _ = writeln!(s, "item --key b back < Back to main menu");
let _ = writeln!(s, "choose target || goto menu");
let _ = writeln!(
s,
"iseq ${{target}} back && chain {base}/boot.ipxe || goto menu"
);
// v0.8.2: `back` is the only fixed item here; every other item value
// is a boot-entry id that can only be resolved at runtime, so dispatch
// on it directly. `echo` is iPXE's rc-0 no-op and keeps a
// non-matching test from aborting the script (issue #2).
let _ = writeln!(s, "iseq ${{target}} back && goto back || echo");
// Pass `?mac=${mac}` so the per-entry handler can record the booting
// client into the Host log. iPXE substitutes `${mac}` before
// the HTTP fetch; if the firmware can't resolve it the literal
// `${mac}` is sent and the server treats it as "unknown".
let _ = writeln!(
s,
"chain {base}/boot/${{target}}.ipxe?mac=${{mac}} || goto menu"
"chain --replace {base}/boot/${{target}}.ipxe?mac=${{mac}} || goto menu"
);
let _ = writeln!(s, ":back");
let _ = writeln!(s, "chain --replace {base}/boot.ipxe || goto menu");
s
}
@@ -331,36 +359,34 @@ pub fn render_tools_menu(isos: &[IsoMeta], base_url: &str) -> String {
let _ = writeln!(s, "item --gap");
let _ = writeln!(s, "item --key b back < Back to main menu");
let _ = writeln!(s, "choose target || goto menu");
let _ = writeln!(
s,
"iseq ${{target}} util && chain {base}/boot/_util.ipxe || goto menu"
);
let _ = writeln!(
s,
"iseq ${{target}} shell && chain {base}/boot/_shell.ipxe || goto menu"
);
let _ = writeln!(
s,
"iseq ${{target}} nic && chain {base}/boot/_nic.ipxe || goto menu"
);
let _ = writeln!(
s,
"iseq ${{target}} reboot && reboot || goto menu"
);
let _ = writeln!(
s,
"iseq ${{target}} firmware && exit 0 || goto menu"
);
let _ = writeln!(
s,
"iseq ${{target}} back && chain {base}/boot.ipxe || goto menu"
);
// v0.8.2: fixed item labels jump to their block; category-Tools ISO
// ids (only known at runtime) fall through to the entry chain. Every
// probe ends in `|| echo` so a non-matching test cannot abort the
// script (issue #2).
let _ = writeln!(s, "iseq ${{target}} util && goto util || echo");
let _ = writeln!(s, "iseq ${{target}} shell && goto shell || echo");
let _ = writeln!(s, "iseq ${{target}} nic && goto nic || echo");
let _ = writeln!(s, "iseq ${{target}} reboot && goto reboot || echo");
let _ = writeln!(s, "iseq ${{target}} firmware && goto firmware || echo");
let _ = writeln!(s, "iseq ${{target}} back && goto back || echo");
// Fall-through for category-Tools ISO ids — same as the family
// submenu, carrying `?mac=${mac}` for the boot log.
let _ = writeln!(
s,
"chain {base}/boot/${{target}}.ipxe?mac=${{mac}} || goto menu"
"chain --replace {base}/boot/${{target}}.ipxe?mac=${{mac}} || goto menu"
);
let _ = writeln!(s, ":util");
let _ = writeln!(s, "chain --replace {base}/boot/_util.ipxe || goto menu");
let _ = writeln!(s, ":shell");
let _ = writeln!(s, "chain --replace {base}/boot/_shell.ipxe || goto menu");
let _ = writeln!(s, ":nic");
let _ = writeln!(s, "chain --replace {base}/boot/_nic.ipxe || goto menu");
let _ = writeln!(s, ":reboot");
let _ = writeln!(s, "reboot || goto menu");
let _ = writeln!(s, ":firmware");
let _ = writeln!(s, "exit 0");
let _ = writeln!(s, ":back");
let _ = writeln!(s, "chain --replace {base}/boot.ipxe || goto menu");
s
}
@@ -384,7 +410,7 @@ pub fn render_local_hdd(base_url: &str) -> String {
);
let _ = writeln!(s, "exit 0");
let _ = writeln!(s, "# If the above exit returns, loop back to the main menu");
let _ = writeln!(s, "chain {base}/boot.ipxe");
let _ = writeln!(s, "chain --replace {base}/boot.ipxe");
s
}
@@ -402,15 +428,18 @@ pub fn render_util(base_url: &str) -> String {
let _ = writeln!(s, "item --gap");
let _ = writeln!(s, "item back < Back");
let _ = writeln!(s, "choose target || goto menu");
// v0.8.2: dispatch to `:memtest`; anything else (only `back` exists
// today) returns to the Tools menu. The old `iseq back && chain
// _tools_menu || goto menu` pair never ran the Tools chain: the
// `back` line's own `iseq` failed first, so `|| goto menu` redrew
// Utilities and Back appeared dead (issue #2).
let _ = writeln!(s, "iseq ${{target}} memtest && goto memtest || echo");
let _ = writeln!(
s,
"iseq ${{target}} memtest && chain {base}/ipxe/memtest.bin || goto menu"
"chain --replace {base}/boot/_tools_menu.ipxe || goto menu"
);
let _ = writeln!(
s,
"iseq ${{target}} back && chain {base}/boot/_tools_menu.ipxe || goto menu"
);
let _ = writeln!(s, "goto menu");
let _ = writeln!(s, ":memtest");
let _ = writeln!(s, "chain --replace {base}/ipxe/memtest.bin || goto menu");
s
}
@@ -425,7 +454,7 @@ pub fn render_shell(base_url: &str) -> String {
let _ = writeln!(s, "echo 'exit' returns to the main menu");
let _ = writeln!(s, "echo ==========================================");
let _ = writeln!(s, "shell");
let _ = writeln!(s, "chain {base}/boot.ipxe");
let _ = writeln!(s, "chain --replace {base}/boot.ipxe");
s
}
@@ -443,8 +472,11 @@ pub fn render_nic_info(base_url: &str) -> String {
let _ = writeln!(s, "route");
let _ = writeln!(s, "echo");
let _ = writeln!(s, "echo 'Press any key to return to menu'");
let _ = writeln!(s, "prompt --timeout 30000");
let _ = writeln!(s, "chain {base}/boot.ipxe");
// v0.8.2: `prompt` returns -ETIMEDOUT when nobody presses a key, and
// a bare failing line aborts the script (dropping the client back to
// firmware). `|| echo` treats the timeout as "return to menu now".
let _ = writeln!(s, "prompt --timeout 30000 || echo");
let _ = writeln!(s, "chain --replace {base}/boot.ipxe");
s
}
@@ -464,7 +496,13 @@ pub fn render_queue_entry(base_url: &str) -> String {
// imgfetch writes the body to a file in iPXE's transient FS; we read
// the queue entry id out of the Location-style header by asking the server
// to put it in the response body as a single token.
let _ = writeln!(s, "chain --replace {base}/api/queue/join?mac=${{mac}}");
// v0.8.2: if the queue endpoint is unreachable, fall back to the main
// menu instead of aborting the whole boot chain into firmware.
let _ = writeln!(
s,
"chain --replace {base}/api/queue/join?mac=${{mac}} \
|| chain --replace {base}/boot.ipxe"
);
s
}
@@ -505,16 +543,21 @@ pub fn render_entry(
cmdline.push_str(extra.trim());
}
}
let _ = writeln!(s, "kernel {base}/{kernel_url} {cmdline}");
// v0.8.2: every fetch line carries its own failure branch.
// Menus now enter boot entries with `chain --replace`, so a
// failing line here can no longer be caught by the *menu's*
// `|| goto menu` — the entry script must return to the menu
// itself.
let _ = writeln!(s, "kernel {base}/{kernel_url} {cmdline} || goto failed");
for u in initrd_urls {
let _ = writeln!(s, "initrd {base}/{u}");
let _ = writeln!(s, "initrd {base}/{u} || goto failed");
}
let _ = writeln!(s, "boot || goto failed");
}
BootKind::Wimboot { wimboot_url, files } => {
let _ = writeln!(s, "kernel {base}/{wimboot_url}");
let _ = writeln!(s, "kernel {base}/{wimboot_url} || goto failed");
for (tag, url) in files {
let _ = writeln!(s, "initrd --name {tag} {base}/{url} {tag}");
let _ = writeln!(s, "initrd --name {tag} {base}/{url} {tag} || goto failed");
}
let _ = writeln!(s, "boot || goto failed");
}
@@ -525,7 +568,7 @@ pub fn render_entry(
let _ = writeln!(s, ":failed");
let _ = writeln!(s, "echo Boot failed - returning to menu in 5s");
let _ = writeln!(s, "sleep 5");
let _ = writeln!(s, "chain {base}/boot.ipxe");
let _ = writeln!(s, "chain --replace {base}/boot.ipxe");
s
}
@@ -589,9 +632,14 @@ pub fn render_password_prompt(entry_id: &str, iso_filename: &str, base_url: &str
let _ = writeln!(s, "echo ==========================================");
let _ = writeln!(s, "set password ");
let _ = writeln!(s, "read --secret password");
// v0.8.2: `isset` (empty → nonzero rc) is the correct emptiness
// test. `iseq ${password} ""` compared against the *literal* token
// `""` — iPXE's tokenizer is whitespace-only and has no quoting, so
// an empty Enter never matched and fell through to the failed-login
// path instead of returning to the menu as the banner promises.
let _ = writeln!(
s,
"iseq ${{password}} \"\" && chain {base}/boot.ipxe || goto submit"
"isset ${{password}} && goto submit || chain --replace {base}/boot.ipxe"
);
let _ = writeln!(s, ":submit");
let _ = writeln!(s, "echo Verifying...");
@@ -601,8 +649,8 @@ pub fn render_password_prompt(entry_id: &str, iso_filename: &str, base_url: &str
// as "unknown" rather than refusing to boot.
let _ = writeln!(
s,
"chain {base}/boot/{entry_id}.ipxe?token=${{password:uristring}}&mac=${{mac}} \
|| chain {base}/boot.ipxe"
"chain --replace {base}/boot/{entry_id}.ipxe?token=${{password:uristring}}&mac=${{mac}} \
|| chain --replace {base}/boot.ipxe"
);
s
}
@@ -621,7 +669,7 @@ pub fn render_password_failed(entry_id: &str, base_url: &str) -> String {
let _ = writeln!(s, "sleep 2");
let _ = writeln!(
s,
"chain {base}/boot/{entry_id}.ipxe || chain {base}/boot.ipxe"
"chain --replace {base}/boot/{entry_id}.ipxe || chain --replace {base}/boot.ipxe"
);
s
}
@@ -640,7 +688,13 @@ mod password_tests {
assert!(s.contains("token=${password:uristring}"));
// Empty enter sends back to the main menu, not back into the prompt
// (avoids a wedged client if the operator chose by mistake).
assert!(s.contains("&& chain http://10.0.0.5/boot.ipxe || goto submit"));
assert!(
s.contains(
"isset ${password} && goto submit \
|| chain --replace http://10.0.0.5/boot.ipxe"
),
"empty-password path missing:\n{s}"
);
// Never log/echo the value.
assert!(!s.contains("echo ${password"));
}
@@ -650,7 +704,7 @@ mod password_tests {
let s = render_password_failed("alpha-linux", "http://10.0.0.5");
assert!(s.contains("Wrong password."));
// Re-target the entry so the prompt flow runs again.
assert!(s.contains("chain http://10.0.0.5/boot/alpha-linux.ipxe"));
assert!(s.contains("chain --replace http://10.0.0.5/boot/alpha-linux.ipxe"));
}
#[test]
@@ -741,3 +795,489 @@ mod password_tests {
}
}
}
/// Control-flow regression tests for the generated iPXE menus.
///
/// Issue #2 ("menu cannot select items") was a script-semantics bug:
/// iPXE evaluates `&&` / `||` strictly left to right, so
/// `iseq ${target} X && chain … || goto menu` runs `goto menu` whenever
/// the `iseq` fails — for every item except the first one. Text
/// assertions cannot catch that, so these tests run the generated
/// scripts through a small interpreter that mirrors the parts of iPXE's
/// `system()` (`src/core/exec.c`) and `process_script()`
/// (`src/image/script.c`) that the menus rely on, and check which URL
/// each menu item actually reaches.
#[cfg(test)]
mod menu_dispatch_tests {
use super::*;
use openpxe_iso_store::{IsoCategory, IsoSource, KernelArgs};
use std::collections::HashMap;
const BASE: &str = "http://10.0.0.5";
/// What a script did before it left the interpreter.
#[derive(Debug, Default)]
struct Run {
/// URLs passed to `chain`, in order (normally exactly one).
chains: Vec<String>,
/// Exit status if the script reached `exit`.
exit: Option<u32>,
/// Whether the script reached `reboot`.
rebooted: bool,
}
#[derive(Clone, Copy)]
enum Sep {
/// `&&`
And,
/// `||`
Or,
/// `;`
Always,
}
enum LineOutcome {
Continue,
Jump(usize),
Done,
Abort(String),
}
enum CmdOutcome {
Rc(i32),
Jump(usize),
Done,
Abort(String),
}
/// Minimal interpreter for the subset of iPXE's script language used
/// by the generated menus.
struct IpxeSim {
vars: HashMap<String, String>,
selection: String,
/// Value the simulated operator types at `read`.
input: String,
/// Model `prompt --timeout` expiring with no key pressed.
prompt_times_out: bool,
run: Run,
}
impl IpxeSim {
fn new(selection: &str) -> Self {
let mut vars = HashMap::new();
vars.insert("buildarch".to_string(), "x86_64".to_string());
vars.insert("platform".to_string(), "efi".to_string());
vars.insert("mac".to_string(), "aa:bb:cc:dd:ee:ff".to_string());
vars.insert("password".to_string(), String::new());
Self {
vars,
selection: selection.to_string(),
input: String::new(),
prompt_times_out: false,
run: Run::default(),
}
}
fn with_var(mut self, name: &str, value: &str) -> Self {
self.vars.insert(name.to_string(), value.to_string());
self
}
fn with_input(mut self, value: &str) -> Self {
self.input = value.to_string();
self
}
/// Run a script to completion and report what it did. Panics if
/// the script aborts or never reaches a terminal command — which
/// is exactly how a bad dispatch line manifests on a real client.
fn run(mut self, script: &str) -> Run {
let lines: Vec<&str> = script.lines().collect();
let mut labels: HashMap<String, usize> = HashMap::new();
for (index, line) in lines.iter().enumerate() {
if let Some(name) = line.trim().strip_prefix(':') {
labels.insert(name.trim().to_string(), index);
}
}
let mut pc = 0;
let mut steps = 0;
while pc < lines.len() {
steps += 1;
assert!(
steps < 10_000,
"script looped without making progress\n--- script ---\n{script}"
);
let line = lines[pc].trim().to_string();
if line.is_empty() || line.starts_with('#') || line.starts_with(':') {
pc += 1;
continue;
}
match self.exec_line(&line, &labels) {
LineOutcome::Continue => pc += 1,
LineOutcome::Jump(target) => pc = target + 1,
LineOutcome::Done => return self.run,
LineOutcome::Abort(why) => {
panic!("script aborted at line {pc}: {why}\n--- script ---\n{script}");
}
}
}
panic!("script ended without booting or exiting\n--- script ---\n{script}");
}
/// Expand `${name}` / `${name:modifier}` from the simulated
/// settings. iPXE expands *after* tokenising, so a value
/// containing spaces stays a single argument.
fn expand(&self, token: &str) -> String {
let mut out = String::new();
let mut rest = token;
while let Some(start) = rest.find("${") {
out.push_str(&rest[..start]);
let after = &rest[start + 2..];
let Some(end) = after.find('}') else {
out.push_str(&rest[start..]);
return out;
};
let spec = &after[..end];
let name = spec.split(':').next().unwrap_or(spec);
if let Some(value) = self.vars.get(name) {
out.push_str(value);
}
rest = &after[end + 1..];
}
out.push_str(rest);
out
}
/// Execute one command line, mirroring `system()`: commands are
/// separated by `&&` / `||` / `;`, and each separator decides
/// whether the *next* command runs based on the last status.
fn exec_line(&mut self, line: &str, labels: &HashMap<String, usize>) -> LineOutcome {
let mut commands: Vec<(Vec<String>, Option<Sep>)> = Vec::new();
let mut current: Vec<String> = Vec::new();
for token in tokenize(line) {
match token.as_str() {
"&&" => commands.push((std::mem::take(&mut current), Some(Sep::And))),
"||" => commands.push((std::mem::take(&mut current), Some(Sep::Or))),
";" => commands.push((std::mem::take(&mut current), Some(Sep::Always))),
_ => current.push(token),
}
}
commands.push((current, None));
let mut rc = 0;
let mut process = true;
for (argv, sep) in &commands {
if process {
if argv.is_empty() {
return LineOutcome::Abort(format!("empty command in `{line}`"));
}
match self.exec_cmd(argv, labels) {
CmdOutcome::Rc(status) => rc = status,
CmdOutcome::Jump(target) => return LineOutcome::Jump(target),
CmdOutcome::Done => return LineOutcome::Done,
CmdOutcome::Abort(why) => return LineOutcome::Abort(why),
}
}
match sep {
Some(Sep::And) => process = rc == 0,
Some(Sep::Or) => process = rc != 0,
Some(Sep::Always) => process = true,
None => break,
}
}
if rc != 0 {
return LineOutcome::Abort(format!("`{line}` failed with status {rc}"));
}
LineOutcome::Continue
}
fn exec_cmd(&mut self, argv: &[String], labels: &HashMap<String, usize>) -> CmdOutcome {
let cmd = self.expand(&argv[0]);
let args: Vec<String> = argv[1..].iter().map(|a| self.expand(a)).collect();
match cmd.as_str() {
"set" => {
if let Some(name) = args.first() {
let value = if args.len() > 1 {
args[1..].join(" ")
} else {
String::new()
};
self.vars.insert(name.clone(), value);
}
CmdOutcome::Rc(0)
}
"iseq" => {
let equal = args.len() >= 2 && args[0] == args[1];
CmdOutcome::Rc(i32::from(!equal))
}
"isset" => {
let nonempty = args.first().is_some_and(|value| !value.is_empty());
CmdOutcome::Rc(i32::from(!nonempty))
}
"goto" => match args.first().and_then(|label| labels.get(label)) {
Some(target) => CmdOutcome::Jump(*target),
None => CmdOutcome::Abort(format!("goto to missing label in `{argv:?}`")),
},
"exit" => {
let code = args
.first()
.and_then(|value| value.parse::<u32>().ok())
.unwrap_or(0);
self.run.exit = Some(code);
CmdOutcome::Done
}
// A boot takes over the machine (and `chain --replace`
// additionally replaces the current script image), so the
// interpreter records the URL and stops.
"chain" | "boot" | "imgexec" | "sanboot" => {
let url = args
.iter()
.find(|arg| !arg.starts_with('-'))
.cloned()
.unwrap_or_default();
self.run.chains.push(url);
CmdOutcome::Done
}
"reboot" => {
self.run.rebooted = true;
CmdOutcome::Done
}
"choose" => {
let mut index = 0;
let mut setting = None;
while index < args.len() {
match args[index].as_str() {
"--default" | "--timeout" | "--key" => index += 2,
option if option.starts_with('-') => index += 1,
name => {
setting = Some(name.to_string());
index += 1;
}
}
}
if let Some(name) = setting {
self.vars.insert(name, self.selection.clone());
}
CmdOutcome::Rc(0)
}
"read" => {
if let Some(name) = args.iter().find(|arg| !arg.starts_with('-')) {
let typed = self.input.clone();
self.vars.insert(name.clone(), typed);
}
CmdOutcome::Rc(0)
}
"prompt" => CmdOutcome::Rc(i32::from(self.prompt_times_out)),
"echo" | "console" | "menu" | "item" | "sleep" | "ifstat" | "route" | "shell"
| "kernel" | "initrd" | "imgfetch" => CmdOutcome::Rc(0),
other => CmdOutcome::Abort(format!("unknown iPXE command `{other}`")),
}
}
}
/// iPXE splits on whitespace only — it has no shell quoting.
fn tokenize(line: &str) -> Vec<String> {
line.split_whitespace().map(str::to_string).collect()
}
// ---- fixtures ----
fn test_iso(id: &str, family: DistroFamily, entries: Vec<BootEntry>) -> IsoMeta {
IsoMeta {
id: id.to_string(),
filename: format!("{id}.iso"),
size_bytes: 700 * 1024 * 1024,
sha256_hex: None,
uploaded_at: time::OffsetDateTime::UNIX_EPOCH,
introspection: openpxe_iso_store::IntrospectionReport {
family,
..Default::default()
},
boot_entries: entries,
source: IsoSource::Local,
password_hash: None,
category: IsoCategory::default(),
}
}
fn linux_entry(id: &str) -> BootEntry {
BootEntry {
id: id.to_string(),
title: format!("Title {id}"),
kind: BootKind::LinuxKernel {
kernel_url: format!("iso/{id}/boot/vmlinuz"),
initrd_urls: vec![format!("iso/{id}/boot/initrd")],
args: KernelArgs::default(),
},
}
}
// ---- top-level menu ----
#[test]
fn every_top_menu_item_reaches_its_own_script() {
let settings = Settings::default();
let menu = render_menu(&[], &settings, BASE);
for (item, expected) in [
("local", "/boot/_local.ipxe"),
("linux", "/boot/_linux_menu.ipxe"),
("windows", "/boot/_windows_menu.ipxe"),
("tools", "/boot/_tools_menu.ipxe"),
("queue", "/boot/_queue.ipxe"),
] {
let run = IpxeSim::new(item).run(&menu);
assert_eq!(
run.chains,
vec![format!("{BASE}{expected}")],
"item `{item}` did not reach its script"
);
assert_eq!(run.exit, None, "item `{item}` should not exit");
}
}
#[test]
fn exit_item_exits_the_firmware() {
let menu = render_menu(&[], &Settings::default(), BASE);
let run = IpxeSim::new("exit").run(&menu);
assert_eq!(run.exit, Some(0));
assert!(run.chains.is_empty(), "{:?}", run.chains);
}
#[test]
fn unknown_architecture_does_not_abort_the_menu() {
// The old arch-label line ended in `|| true`, which is not an
// iPXE command: any arch/platform pair outside the allowlist
// aborted the whole script. `|| echo` keeps the generic label.
for (arch, platform) in [("riscv64", "efi"), ("arm64", "pcbios"), ("i386", "efi")] {
let run = IpxeSim::new("local")
.with_var("buildarch", arch)
.with_var("platform", platform)
.run(&render_menu(&[], &Settings::default(), BASE));
assert_eq!(run.chains.len(), 1, "menu aborted on {arch}/{platform}");
}
}
// ---- family (installer) menu ----
#[test]
fn family_menu_dispatches_each_entry_and_back() {
let isos = vec![
test_iso(
"alpha",
DistroFamily::DebianUbuntu,
vec![linux_entry("alpha-linux")],
),
test_iso(
"beta",
DistroFamily::Alpine,
vec![linux_entry("beta-linux")],
),
];
let menu = render_family_menu(&isos, BASE, false);
for id in ["alpha-linux", "beta-linux"] {
let run = IpxeSim::new(id).run(&menu);
assert_eq!(
run.chains,
vec![format!("{BASE}/boot/{id}.ipxe?mac=aa:bb:cc:dd:ee:ff")],
"entry `{id}` fell through to the menu"
);
}
let run = IpxeSim::new("back").run(&menu);
assert_eq!(run.chains, vec![format!("{BASE}/boot.ipxe")]);
}
// ---- tools menu ----
#[test]
fn tools_menu_dispatches_every_item() {
let mut iso = test_iso(
"tool",
DistroFamily::Unknown,
vec![linux_entry("tool-disk")],
);
iso.category = IsoCategory::Tools;
let menu = render_tools_menu(&[iso], BASE);
for (item, expected) in [
("util", format!("{BASE}/boot/_util.ipxe")),
("shell", format!("{BASE}/boot/_shell.ipxe")),
("nic", format!("{BASE}/boot/_nic.ipxe")),
("back", format!("{BASE}/boot.ipxe")),
(
"tool-disk",
format!("{BASE}/boot/tool-disk.ipxe?mac=aa:bb:cc:dd:ee:ff"),
),
] {
let run = IpxeSim::new(item).run(&menu);
assert_eq!(run.chains, vec![expected], "item `{item}`");
}
let run = IpxeSim::new("reboot").run(&menu);
assert!(run.rebooted, "reboot item did not reboot");
let run = IpxeSim::new("firmware").run(&menu);
assert_eq!(run.exit, Some(0), "firmware item did not exit");
}
// ---- utilities menu ----
#[test]
fn util_menu_dispatches_memtest_and_back() {
let menu = render_util(BASE);
let run = IpxeSim::new("memtest").run(&menu);
assert_eq!(run.chains, vec![format!("{BASE}/ipxe/memtest.bin")]);
let run = IpxeSim::new("back").run(&menu);
assert_eq!(run.chains, vec![format!("{BASE}/boot/_tools_menu.ipxe")]);
}
// ---- small scripts reached from the menus ----
#[test]
fn shell_and_queue_scripts_end_in_their_chain() {
let run = IpxeSim::new("").run(&render_shell(BASE));
assert_eq!(run.chains, vec![format!("{BASE}/boot.ipxe")]);
let run = IpxeSim::new("").run(&render_queue_entry(BASE));
assert_eq!(
run.chains,
vec![format!("{BASE}/api/queue/join?mac=aa:bb:cc:dd:ee:ff")]
);
}
#[test]
fn nic_info_prompt_timeout_returns_to_menu() {
let mut sim = IpxeSim::new("");
sim.prompt_times_out = true;
let run = sim.run(&render_nic_info(BASE));
assert_eq!(run.chains, vec![format!("{BASE}/boot.ipxe")]);
}
#[test]
fn password_prompt_empty_enter_returns_to_main_menu() {
let menu = render_password_prompt("alpha-linux", "Alpha.iso", BASE);
let run = IpxeSim::new("").run(&menu);
assert_eq!(run.chains, vec![format!("{BASE}/boot.ipxe")]);
let run = IpxeSim::new("").with_input("hunter2").run(&menu);
assert_eq!(
run.chains,
vec![format!(
"{BASE}/boot/alpha-linux.ipxe?token=hunter2&mac=aa:bb:cc:dd:ee:ff"
)]
);
}
#[test]
fn boot_entry_scripts_recover_to_the_menu_on_failed_fetches() {
// Menus now chain entries with `--replace`, so the entry script —
// not the caller — has to recover from a failed kernel/initrd
// fetch. Otherwise a flaky mirror drops the client to firmware.
let settings = Settings::default();
let s = render_entry(&linux_entry("alpha-linux"), &settings, BASE, None);
for line in s
.lines()
.filter(|line| line.starts_with("kernel ") || line.starts_with("initrd "))
{
assert!(
line.ends_with("|| goto failed"),
"missing failure branch on `{line}`"
);
}
assert!(s.contains("chain --replace http://10.0.0.5/boot.ipxe"));
}
}
+1
View File
@@ -17,6 +17,7 @@
pub mod app;
pub mod auth;
pub mod error;
pub mod fetch;
pub mod grub_script;
pub mod ipxe_script;
pub mod log_stream;
@@ -1,5 +1,6 @@
---
source: crates/http-api/src/ipxe_script.rs
assertion_line: 787
expression: rendered
---
#!ipxe
@@ -9,7 +10,9 @@ set esc:hex 1b
set cls ${esc:string}[2J
iseq ${platform} efi && console --picture http://10.0.0.5/branding/pxe-logo --top 290 || console
set arch-label ${buildarch} ${platform}
iseq ${buildarch} i386 && iseq ${platform} pcbios && set arch-label x86 BIOS || iseq ${buildarch} x86_64 && iseq ${platform} efi && set arch-label x86_64 UEFI || iseq ${buildarch} arm64 && iseq ${platform} efi && set arch-label arm64 UEFI || true
iseq ${buildarch} i386 && iseq ${platform} pcbios && set arch-label x86 BIOS || echo
iseq ${buildarch} x86_64 && iseq ${platform} efi && set arch-label x86_64 UEFI || echo
iseq ${buildarch} arm64 && iseq ${platform} efi && set arch-label arm64 UEFI || echo
:menu
menu OpenPXE - network boot menu
item --gap
@@ -27,10 +30,16 @@ item --key x exit Exit iPXE
item --gap
item --gap -- OpenPXE vX.Y.Z - ${arch-label}
choose --default queue --timeout 600000 target || goto menu
iseq ${target} local && chain http://10.0.0.5/boot/_local.ipxe || goto menu
iseq ${target} linux && chain http://10.0.0.5/boot/_linux_menu.ipxe || goto menu
iseq ${target} windows && chain http://10.0.0.5/boot/_windows_menu.ipxe || goto menu
iseq ${target} tools && chain http://10.0.0.5/boot/_tools_menu.ipxe || goto menu
iseq ${target} queue && chain http://10.0.0.5/boot/_queue.ipxe || goto menu
iseq ${target} exit && exit || goto menu
goto menu
goto ${target} || goto menu
:local
chain --replace http://10.0.0.5/boot/_local.ipxe || goto menu
:linux
chain --replace http://10.0.0.5/boot/_linux_menu.ipxe || goto menu
:windows
chain --replace http://10.0.0.5/boot/_windows_menu.ipxe || goto menu
:tools
chain --replace http://10.0.0.5/boot/_tools_menu.ipxe || goto menu
:queue
chain --replace http://10.0.0.5/boot/_queue.ipxe || goto menu
:exit
exit 0
+5
View File
@@ -1,4 +1,5 @@
use crate::auth::SessionStore;
use crate::fetch::FetchJobs;
use crate::saml_routes::SamlRuntime;
use crate::uploads::UploadSessions;
use openpxe_core::{
@@ -108,6 +109,10 @@ pub struct AppState {
/// through `IsoStore`, but the UI uses sessions so large ISO transfers
/// can show deterministic progress and leave visible partial files.
pub uploads: UploadSessions,
/// v0.8.1: server-side "add ISO by URL" jobs. Background downloads
/// streamed straight into the store (reusing the upload handle +
/// introspection); the Storage view polls their progress.
pub fetch_jobs: FetchJobs,
/// Live log bus consumed by the Terminal tab via SSE. Operator-issued
/// terminal commands also push synthetic lines onto it so the tail
/// shows them inline.
+52 -2
View File
@@ -126,6 +126,7 @@ async fn build_state() -> (AppState, tempfile::TempDir) {
sftp_shares,
unattended,
uploads: openpxe_http_api::uploads::UploadSessions::default(),
fetch_jobs: openpxe_http_api::fetch::FetchJobs::default(),
log_bus,
started_at: time::OffsetDateTime::now_utc(),
public_base_url: "http://127.0.0.1".into(),
@@ -191,6 +192,55 @@ async fn api_key_authenticates_gated_endpoints() {
assert_eq!(res.status(), StatusCode::OK, "valid key must authenticate");
}
#[tokio::test]
async fn fetch_iso_by_url_downloads_into_store() {
// v0.8.1: add-ISO-by-URL. Serve a real ISO over HTTP, POST its URL, poll
// the fetch registry until the background download finishes, then assert
// the image landed in the store (classified like an upload).
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};
let (state, _dir) = build_state().await;
let app = build_router(state);
let iso = fake_alpine_iso();
let server = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/rescue.iso"))
.respond_with(ResponseTemplate::new(200).set_body_bytes(iso))
.mount(&server)
.await;
let url = format!("{}/rescue.iso", server.uri());
let (code, _) = post_json(&app, "/api/isos/fetch", &format!(r#"{{"url":"{url}"}}"#)).await;
assert_eq!(code, StatusCode::ACCEPTED, "fetch should start");
// Bounded poll for completion (the download runs on a spawned task).
let mut done = false;
for _ in 0..100 {
let (_, body) = get(&app, "/api/isos/fetch").await;
let v: serde_json::Value = serde_json::from_slice(&body).unwrap();
let jobs = v["jobs"].as_array().cloned().unwrap_or_default();
assert!(
!jobs.iter().any(|j| j["state"] == "failed"),
"fetch failed: {jobs:?}"
);
if jobs.iter().any(|j| j["state"] == "done") {
done = true;
break;
}
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
}
assert!(done, "fetch did not complete in time");
// The fetched ISO is now in the store under the URL basename.
let (_, body) = get(&app, "/api/isos").await;
assert!(
String::from_utf8_lossy(&body).contains("rescue.iso"),
"fetched ISO should appear in /api/isos"
);
}
#[tokio::test]
async fn health_and_ready_endpoints() {
let (state, _dir) = build_state().await;
@@ -445,11 +495,11 @@ async fn reboot_and_firmware_exit_in_tools_menu() {
"tools menu missing firmware-exit item:\n{tools}"
);
assert!(
tools.contains("&& reboot"),
tools.contains(":reboot\nreboot || goto menu"),
"reboot command not wired:\n{tools}"
);
assert!(
tools.contains("&& exit 0"),
tools.contains(":firmware\nexit 0"),
"firmware exit command not wired:\n{tools}"
);
}
+33
View File
@@ -120,6 +120,38 @@ async fn main() -> anyhow::Result<()> {
let boot_log = openpxe_core::BootLog::load_or_default(&config.paths.work_dir);
let branding = openpxe_core::BrandingStore::load_or_default(&config.paths.work_dir);
let admin = openpxe_core::AdminStore::load_or_default(&config.paths.work_dir);
// v0.8.1: zero-touch first-run bootstrap. If no admin exists yet and the
// operator supplied OPENPXE_ADMIN_USERNAME + OPENPXE_ADMIN_PASSWORD (or
// …_PASSWORD_FILE, for Docker/K8s secrets), create the admin now so a
// fresh container is usable without the web setup wizard. Seeds the
// first run only — once an admin exists (including one made in the UI)
// this is a no-op, so a lingering env var can't reset a rotated password.
if !admin.is_configured() {
if let Ok(username) = std::env::var("OPENPXE_ADMIN_USERNAME") {
let password = std::env::var("OPENPXE_ADMIN_PASSWORD_FILE")
.ok()
.and_then(|p| std::fs::read_to_string(p).ok())
.map(|s| s.trim_end_matches(['\n', '\r']).to_string())
.or_else(|| std::env::var("OPENPXE_ADMIN_PASSWORD").ok());
if let Some(password) = password {
match admin.bootstrap(&username, &password) {
Ok(p) => tracing::info!(
target: "openpxe::auth", username = %p.username,
"admin bootstrapped from environment"
),
Err(e) => tracing::warn!(
target: "openpxe::auth",
"env admin bootstrap failed ({e}); use the web setup wizard"
),
}
} else {
tracing::warn!(
target: "openpxe::auth",
"OPENPXE_ADMIN_USERNAME set without OPENPXE_ADMIN_PASSWORD[_FILE]; skipping bootstrap"
);
}
}
}
let sso = openpxe_core::SsoStore::load_or_default(&config.paths.work_dir);
let notify = openpxe_core::NotifyStore::load_or_default(&config.paths.work_dir);
let api_key = openpxe_core::ApiKeyStore::load_or_init(&config.paths.work_dir);
@@ -213,6 +245,7 @@ async fn main() -> anyhow::Result<()> {
sftp_shares: sftp_shares.clone(),
unattended: unattended.clone(),
uploads: openpxe_http_api::uploads::UploadSessions::default(),
fetch_jobs: openpxe_http_api::fetch::FetchJobs::default(),
log_bus: log_bus.clone(),
started_at: time::OffsetDateTime::now_utc(),
public_base_url: public_base_url.clone(),
+75 -2
View File
@@ -768,6 +768,75 @@
}
}
// v0.8.1: add ISO by URL. Paste a link and the server streams it
// straight into the store and auto-detects it — no download-then-
// reupload. Progress polls /api/isos/fetch and shows rows below,
// mirroring uploads. On an air-gapped network, use the drop zone.
const urlInput = el('input', {type:'url', id:'iso-url', style:'flex:1',
placeholder:'https://example.com/systemrescue.iso'});
const fetchBtn = el('button', {class:'ghost', type:'button', style:'margin-left:8px'}, 'Fetch');
const fetchMsg = el('div', {class:'msg', style:'margin-top:6px'});
const fetchList = el('div', {id:'fetches', style:'display:grid;gap:12px;margin-top:12px'});
const urlRow = el('div', {style:'margin-top:14px'}, [
el('label', {class:'field', style:'margin-bottom:0'}, [
el('span', {class:'name'}, 'Or add by URL'),
el('div', {style:'display:flex;align-items:center'}, [urlInput, fetchBtn]),
el('span', {class:'hint'},
'The server downloads the .iso into storage and auto-detects it — same result as a drag-drop. Any http(s) .iso link works.'),
]),
fetchMsg,
]);
let fetchTimer = null;
const renderFetchRows = (jobs) => {
fetchList.replaceChildren(...jobs.map(j => {
const pct = j.total > 0 ? Math.min(100, (j.downloaded / j.total) * 100)
: (j.state === 'done' ? 100 : 0);
let text, cls = '';
if (j.state === 'downloading')
text = 'Downloading ' + fmtBytes(j.downloaded) +
(j.total ? ' of ' + fmtBytes(j.total) + ' (' + pct.toFixed(0) + '%)' : '');
else if (j.state === 'done') { text = 'Downloaded and analyzed.'; cls = 'ok'; }
else if (j.state === 'failed') { text = 'Failed: ' + (j.error || 'unknown error'); cls = 'err'; }
else text = 'Canceled — partial discarded.';
const btn = el('button', {class:'danger', type:'button', style:'margin-top:8px'},
j.state === 'downloading' ? 'Cancel' : 'Dismiss');
btn.onclick = async () => {
try { await fetch('/api/isos/fetch/' + encodeURIComponent(j.id), {method:'DELETE'}); } catch (_) {}
pollFetches();
};
return el('div', {}, [
el('div', {style:'font-weight:600;font-size:13px;margin-bottom:6px;word-break:break-all'},
j.filename + ' · ' + j.url),
el('div', {class:'progress' + (j.state === 'downloading' ? ' active' : '')},
el('div', {class:'bar', style:'width:' + pct.toFixed(1) + '%'})),
el('div', {class:'msg ' + cls}, text),
btn,
]);
}));
};
async function pollFetches() {
if (fetchTimer) { clearTimeout(fetchTimer); fetchTimer = null; }
let jobs = [];
try { jobs = (await getJSON('/api/isos/fetch')).jobs || []; } catch (_) {}
renderFetchRows(jobs);
// A successful fetch is read-once on the server, so refreshing here
// shows the new image and won't re-trigger on the next poll. Keep
// polling only while a download is still in flight.
if (jobs.some(j => j.state === 'done')) { render('storage'); return; }
if (jobs.some(j => j.state === 'downloading')) fetchTimer = setTimeout(pollFetches, 1500);
}
async function startFetch() {
const url = urlInput.value.trim();
if (!url) return;
fetchMsg.textContent = 'Starting…'; fetchMsg.className = 'msg';
const r = await postJSON('/api/isos/fetch', { url });
if (r.ok) { urlInput.value = ''; fetchMsg.textContent = ''; pollFetches(); }
else { fetchMsg.textContent = 'Could not start: ' + (await r.text()).slice(0, 160); fetchMsg.className = 'msg err'; }
}
fetchBtn.onclick = startFetch;
urlInput.addEventListener('keydown', e => { if (e.key === 'Enter') { e.preventDefault(); startFetch(); } });
// ── ISO table (mixed local + SMB) ──
// Each row gets a "Password" cell that toggles a small inline
// editor (a checkbox + a password field + Save button) inside the
@@ -1397,11 +1466,11 @@
]),
]);
return el('div', {}, [el('div', {class:'grid'}, [
const root = el('div', {}, [el('div', {class:'grid'}, [
diskCard,
el('div', {class:'card'}, [
el('header', {}, el('h2', {}, 'Upload ISO')),
el('div', {class:'body'}, [drop, file, uploadsList]),
el('div', {class:'body'}, [drop, file, urlRow, uploadsList, fetchList]),
]),
// v0.5.1: SMB + NFS unified into one "Remote shares" card with a
// protocol dropdown. Backend endpoints are unchanged; this is a
@@ -1440,6 +1509,10 @@
isoPager,
]),
]), unattendedAdvanced]);
// v0.8.1: resume/kick URL-fetch progress polling; stop it on view swap.
root._cleanup = () => { if (fetchTimer) clearTimeout(fetchTimer); };
pollFetches();
return root;
},
hosts: async () => {