Compare commits

...
4 Commits
Author SHA1 Message Date
Miles Ward a171331a7a make container builds reproducible
Commit Cargo.lock, copy it into the Docker build stage, and align the Docker Rust base/MSRV with the toolchain required by the locked dependency graph.
2026-05-24 13:53:10 -04:00
Miles Ward fe4a127422 fix docker build toolchain selection
Do not copy rust-toolchain.toml into the Docker build stage so the release image uses the Rust toolchain provided by the base image instead of downloading latest stable inside the container.
2026-05-24 13:49:09 -04:00
Miles Ward 2c1c80a7ca v0.4.1: harden ISO uploads and beta UI polish
Add browser-safe chunked ISO uploads with progress, partial-file visibility, offset validation, and abort cleanup while keeping the legacy multipart endpoint for API clients.

Record host-log validation coverage, keep the queue/status UI copy clean, move release docs to 0.4.1, and tighten the dark theme to a near-black Netbox-style palette.
2026-05-24 13:45:35 -04:00
Miles WardandClaude Opus 4.7 ec171ede47 v0.4.0: upload telemetry, host log, jet-black UI
- Upload reliability + diagnostics:
  - api_upload_iso now distinguishes clean EOF from mid-stream errors;
    a truncated multipart body (proxy buffer cap, network drop) returns
    400 with the cause and a "try the LAN IP" hint instead of silently
    finalising a partial file.
  - Per-stage tracing (begin/MB-watermark/finish/abort) so a stuck
    upload is debuggable from the Terminal tab.
  - Web upload UI surfaces bytes/total, percent, throughput, ETA, and
    maps 413/502/504/network-drop to actionable hints.
- New BootLog feature under Hosts:
  - openpxe-core::BootLog — bounded in-memory ring (500) + append-only
    JSONL on disk, recording (timestamp, mac, ip, target_id,
    target_title) every time a boot entry script is served.
  - iPXE per-entry chain URLs grow ?mac=${mac}; password prompt
    submission carries it through; host-binding short-circuit uses the
    bound MAC. ConnectInfo<SocketAddr> wired for peer IP capture (with
    optional fallback so tower::oneshot in tests still works).
  - GET /api/boot-log endpoint + Host log table under the Hosts tab.
- UI changes:
  - Queue card header "Forge" → "Status".
  - Removed Tinkerbell attribution sentence from Hosts tab.
  - Topbar readiness chip moved into the sidebar footer as
    "Service status: Ready / Advertised to clients / <url>", grouping
    advertised PXE URL with operator-relevant status.
  - Jet-black dark palette (#000 / #0a0a0a / #141414 / #1c1c1c)
    replacing the blue-tinted ramp; terminal toolbar/input recoloured
    to match.
- 89 tests passing (was 85 in v0.3.2); cargo clippy --workspace
  --all-targets clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-24 13:10:40 -04:00
24 changed files with 3732 additions and 129 deletions
-1
View File
@@ -1,5 +1,4 @@
/target
Cargo.lock
data/isos/*.iso
data/isos/*.partial
data/isos/*.meta.json
Generated
+2438
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -12,9 +12,9 @@ members = [
]
[workspace.package]
version = "0.3.2"
version = "0.4.1"
edition = "2021"
rust-version = "1.80"
rust-version = "1.95"
license = "MIT OR Apache-2.0"
repository = "https://gitea.milesward.dev/mward4/OpenPXE"
authors = ["OpenPXE contributors"]
+10 -11
View File
@@ -5,11 +5,11 @@ Container-native PXE boot server. A Rust reimplementation of
for Docker/OCI and OpenShift. Upload `.iso` files via the web UI; network
clients PXE-boot them.
> **Status:** v0.3.2 / pre-beta. Phases 15 complete: full PXE stack,
> **Status:** v0.4.1 / pre-beta. Phases 15 complete: full PXE stack,
> Queued Deployment queue, NFS-share ISO sources, live tracing log + an
> operator terminal, per-MAC host bindings (Tinkerbell-style),
> Prometheus `/metrics`, light/dark theme toggle, animated OpenPXE
> imaging-progress widget, and per-ISO boot passwords. The test suite and
> operator terminal, per-MAC host bindings, Prometheus `/metrics`,
> light/dark theme toggle, animated OpenPXE imaging-progress widget,
> chunked ISO uploads, and per-ISO boot passwords. The test suite and
> clippy are part of the release checklist. Ready for real-hardware validation.
## Design non-negotiables
@@ -51,8 +51,7 @@ clients PXE-boot them.
widget when devices are imaging. All assets served from the binary —
no external requests.
8. **Per-MAC host bindings.** Pin a MAC to a boot target and the client
skips the menu, chains straight through. Inspired by Tinkerbell's
`smee` MAC-prepended URL pattern.
skips the menu, chains straight through.
9. **Prometheus metrics** at `/metrics` — DHCP replies by arch, TFTP
transfer counts and bytes, HTTP request counts by route, queue /
imaging gauges, uptime, build info. Plain text exposition format,
@@ -81,7 +80,7 @@ skip TFTP and respond with an HTTP URL.
./scripts/fetch-ipxe.sh
# 2. Build the container image (~3 min first time).
docker buildx build -f deploy/docker/Dockerfile -t openpxe:0.3.2 --load .
docker buildx build -f deploy/docker/Dockerfile -t openpxe:0.4.1 --load .
# 3. Run it on the box plugged into your PXE network. Set PUBLIC_IP to
# this host's LAN address so advertised iPXE URLs are reachable.
@@ -91,7 +90,7 @@ docker run -d --name openpxe \
-e OPENPXE_DHCP_MODE=proxy \
-v $PWD/data/isos:/var/lib/openpxe/isos \
-v $PWD/data/work:/var/lib/openpxe/work \
openpxe:0.3.2
openpxe:0.4.1
# 4. Open the UI and drop an ISO in.
open http://10.0.0.5
@@ -122,7 +121,7 @@ docker buildx create --name openpxe-multi --driver docker-container --use
# Build + push both linux/amd64 and linux/arm64 under one tag.
docker buildx build --builder openpxe-multi \
--platform linux/amd64,linux/arm64 \
-t ghcr.io/YOUR-ORG/openpxe:0.3.2 \
-t ghcr.io/YOUR-ORG/openpxe:0.4.1 \
--push \
-f deploy/docker/Dockerfile .
```
@@ -155,10 +154,10 @@ docker run --rm \
-v /my/iso-library:/seed:ro \
-v openpxe-data:/var/lib/openpxe/isos \
-e OPENPXE_PUBLIC_IP=10.0.0.5 \
openpxe:0.3.2 seed --from /seed
openpxe:0.4.1 seed --from /seed
# Dry run first to see what would be imported:
docker run --rm -v /my/iso-library:/seed:ro openpxe:0.3.2 seed --from /seed --dry-run
docker run --rm -v /my/iso-library:/seed:ro openpxe:0.4.1 seed --from /seed --dry-run
```
### Environment overrides
+249
View File
@@ -0,0 +1,249 @@
//! Boot-event log — "who installed what, when, from where".
//!
//! Each `/boot/<entry>.ipxe` fetch that actually goes on to serve a boot
//! script lands an entry here. The log is bounded in memory (newest-first,
//! ring-buffered at [`BootLog::CAP`]) and is mirrored append-only to
//! `<work_dir>/boot_log.jsonl`. Mirrors `HostBindings`'s "in-memory is
//! authoritative, disk is a cache" policy — a corrupt log file should
//! never block PXE for the network.
//!
//! We deliberately don't push these onto the `LogBus` (the operator
//! terminal stream). The terminal already shows the http traces; the
//! Host log is a curated, persistent, easy-to-scan view of "what got
//! imaged on what hardware" and conflating the two would be noisy.
use parking_lot::RwLock;
use serde::{Deserialize, Serialize};
use std::collections::VecDeque;
use std::io::Write;
use std::net::IpAddr;
use std::path::PathBuf;
use std::sync::Arc;
use time::OffsetDateTime;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BootEvent {
#[serde(with = "time::serde::rfc3339")]
pub timestamp: OffsetDateTime,
/// Lowercase, colon-separated. `None` when iPXE didn't supply
/// `?mac=${mac}` in the chain URL (older bookmarks, custom scripts).
pub mac: Option<String>,
/// Connecting peer's IP — taken from the TCP socket when available
/// (PXE clients connect direct, no reverse proxy), and falls back to
/// `X-Forwarded-For` for the rare case where one is present.
pub ip: Option<IpAddr>,
/// `BootEntry::id` — the same id used in `/boot/<id>.ipxe`.
pub target_id: String,
/// Human-friendly label: the ISO's filename / volume label / entry
/// title. Pre-resolved at log time so the UI can render without
/// joining against the ISO store (and so "what image was installed?"
/// survives the operator deleting the ISO later).
pub target_title: String,
}
/// In-memory ring + disk-backed append log of boot events. Cheap to
/// clone; the inner state is `Arc<RwLock<_>>`.
#[derive(Debug, Clone)]
pub struct BootLog {
path: Arc<PathBuf>,
inner: Arc<RwLock<VecDeque<BootEvent>>>,
}
impl BootLog {
/// Newest entries we retain in memory. Past this, the oldest gets
/// evicted — the on-disk JSONL keeps the full history for offline
/// inspection. 500 covers a typical install-day's worth without
/// turning the Hosts tab into a wall of text.
pub const CAP: usize = 500;
/// Load up to `CAP` newest events from `<work_dir>/boot_log.jsonl`,
/// or start empty if the file is missing / unreadable.
#[must_use]
pub fn load_or_default(work_dir: &std::path::Path) -> Self {
let path = work_dir.join("boot_log.jsonl");
let mut events = VecDeque::with_capacity(Self::CAP);
if let Ok(text) = std::fs::read_to_string(&path) {
for line in text.lines() {
if line.trim().is_empty() {
continue;
}
match serde_json::from_str::<BootEvent>(line) {
Ok(ev) => {
if events.len() == Self::CAP {
events.pop_front();
}
events.push_back(ev);
}
Err(e) => {
tracing::warn!(
target: "openpxe::boot_log",
"skipping unparseable boot_log line: {e}"
);
}
}
}
}
Self {
path: Arc::new(path),
inner: Arc::new(RwLock::new(events)),
}
}
/// Append an event. Persistence is best-effort and never blocks the
/// caller on a failed write (the in-memory copy is the source of
/// truth for the live UI; the JSONL is just for crash survival).
pub fn record(&self, ev: &BootEvent) {
// Push into the ring first so a slow / failing disk doesn't lose
// events for the live UI.
{
let mut g = self.inner.write();
if g.len() == Self::CAP {
g.pop_front();
}
g.push_back(ev.clone());
}
tracing::info!(
target: "openpxe::boot_log",
mac = ev.mac.as_deref().unwrap_or("?"),
ip = ev.ip.map(|i| i.to_string()).as_deref().unwrap_or("?"),
target = %ev.target_id,
"boot event"
);
// Append to disk. We tolerate write failures — they'd show up as
// missing entries on the next restart only.
let mut line = match serde_json::to_string(ev) {
Ok(s) => s,
Err(e) => {
tracing::warn!(target: "openpxe::boot_log", "serialize boot event: {e}");
return;
}
};
line.push('\n');
if let Some(parent) = self.path.parent() {
let _ = std::fs::create_dir_all(parent);
}
match std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(self.path.as_path())
{
Ok(mut f) => {
if let Err(e) = f.write_all(line.as_bytes()) {
tracing::warn!(target: "openpxe::boot_log", "append boot_log.jsonl: {e}");
}
}
Err(e) => {
tracing::warn!(target: "openpxe::boot_log", "open boot_log.jsonl: {e}");
}
}
}
/// Newest-first snapshot, up to `CAP` entries.
#[must_use]
pub fn list(&self) -> Vec<BootEvent> {
let g = self.inner.read();
// VecDeque preserves insertion order; reverse so newest is first.
g.iter().rev().cloned().collect()
}
#[must_use]
pub fn len(&self) -> usize {
self.inner.read().len()
}
#[must_use]
pub fn is_empty(&self) -> bool {
self.len() == 0
}
/// Wipe in-memory + the on-disk file. Used by the `terminal clear`
/// equivalent or future operator action; not currently wired to a UI
/// button but exposed for completeness.
pub fn clear(&self) {
self.inner.write().clear();
let _ = std::fs::remove_file(self.path.as_path());
}
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::tempdir;
fn ev(target: &str, mac: Option<&str>) -> BootEvent {
BootEvent {
timestamp: OffsetDateTime::now_utc(),
mac: mac.map(str::to_string),
ip: Some("10.0.0.42".parse().unwrap()),
target_id: target.into(),
target_title: format!("{target}.iso"),
}
}
#[test]
fn record_then_list_is_newest_first() {
let dir = tempdir().unwrap();
let log = BootLog::load_or_default(dir.path());
assert!(log.is_empty());
log.record(&ev("alpha", Some("aa:bb:cc:00:00:01")));
log.record(&ev("beta", Some("aa:bb:cc:00:00:02")));
let list = log.list();
assert_eq!(list.len(), 2);
assert_eq!(list[0].target_id, "beta");
assert_eq!(list[1].target_id, "alpha");
}
#[test]
fn round_trip_through_disk() {
let dir = tempdir().unwrap();
let log = BootLog::load_or_default(dir.path());
log.record(&ev("alpha", Some("aa:bb:cc:00:00:01")));
log.record(&ev("beta", None));
drop(log);
let log2 = BootLog::load_or_default(dir.path());
assert_eq!(log2.len(), 2);
let list = log2.list();
assert_eq!(list[0].target_id, "beta");
assert_eq!(list[1].target_id, "alpha");
assert!(list[0].mac.is_none());
assert_eq!(list[1].mac.as_deref(), Some("aa:bb:cc:00:00:01"));
}
#[test]
fn ring_evicts_oldest_past_cap() {
let dir = tempdir().unwrap();
let log = BootLog::load_or_default(dir.path());
for i in 0..(BootLog::CAP + 5) {
log.record(&ev(&format!("e{i}"), None));
}
assert_eq!(log.len(), BootLog::CAP);
let list = log.list();
// Newest first; the most recent push is the last index inserted.
assert_eq!(list[0].target_id, format!("e{}", BootLog::CAP + 4));
// Oldest in-memory should be the 6th push (0..5 were evicted).
assert_eq!(list[BootLog::CAP - 1].target_id, "e5");
}
#[test]
fn clear_wipes_memory_and_disk() {
let dir = tempdir().unwrap();
let log = BootLog::load_or_default(dir.path());
log.record(&ev("alpha", None));
log.clear();
assert!(log.is_empty());
let log2 = BootLog::load_or_default(dir.path());
assert!(log2.is_empty());
}
#[test]
fn corrupt_disk_lines_are_skipped_not_fatal() {
// Write a file with one valid + one garbage line; loader should
// surface the valid one and skip the garbage.
let dir = tempdir().unwrap();
let path = dir.path().join("boot_log.jsonl");
let valid = serde_json::to_string(&ev("ok", Some("aa:bb:cc:00:00:09"))).unwrap();
std::fs::write(&path, format!("{valid}\nNOT_JSON\n{valid}\n")).unwrap();
let log = BootLog::load_or_default(dir.path());
assert_eq!(log.len(), 2);
}
}
+4 -5
View File
@@ -1,10 +1,9 @@
//! Per-MAC host bindings.
//!
//! Inspired by the Tinkerbell `smee` "MAC-prepended URL" pattern: an
//! operator can attach a preferred boot target (a `BootEntry::id`) to a
//! specific MAC address. When a client with that MAC arrives, the
//! top-level boot script chains straight to that target instead of
//! showing the interactive menu.
//! Operators can attach a preferred boot target (a `BootEntry::id`) to a
//! specific MAC address. When a client with that MAC arrives, the top-level
//! boot script chains straight to that target instead of showing the
//! interactive menu.
//!
//! Use cases:
//! - "This rack of Dell servers always images with Ubuntu Server 24.04"
+2
View File
@@ -3,6 +3,7 @@
#![forbid(unsafe_code)]
pub mod arch;
pub mod boot_log;
pub mod client;
pub mod config;
pub mod error;
@@ -13,6 +14,7 @@ pub mod queue;
pub mod settings;
pub use arch::{ClientArch, FirmwareClass};
pub use boot_log::{BootEvent, BootLog};
pub use client::{ClientEvent, ClientRegistry, ClientSnapshot};
pub use config::{Config, DhcpMode, NetworkConfig, Paths, ServerConfig};
pub use error::{Error, Result};
+1
View File
@@ -31,6 +31,7 @@ bytes.workspace = true
futures.workspace = true
mime.workspace = true
mime_guess.workspace = true
uuid.workspace = true
[dev-dependencies]
tokio = { workspace = true, features = ["macros", "rt", "rt-multi-thread", "time"] }
+358 -30
View File
@@ -22,18 +22,19 @@ use crate::log_stream;
use crate::state::AppState;
use crate::terminal;
use axum::{
body::Body,
extract::{DefaultBodyLimit, Multipart, Path as AxumPath, Query, State},
body::{Body, Bytes},
extract::{ConnectInfo, DefaultBodyLimit, Multipart, Path as AxumPath, Query, State},
http::{header, HeaderMap, HeaderValue, StatusCode},
response::{IntoResponse, Response},
routing::{delete, get, post},
routing::{delete, get, post, put},
Json, Router,
};
use openpxe_core::{ClientEvent, Settings};
use openpxe_core::{BootEvent, ClientEvent, Error, Settings};
use openpxe_ipxe_assets::asset_bytes;
use openpxe_iso_store::{IsoMeta, NfsAddRequest};
use serde::Deserialize;
use serde_json::json;
use std::net::SocketAddr;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncSeekExt};
use tower_http::trace::TraceLayer;
@@ -61,6 +62,11 @@ pub fn build_router(state: AppState) -> Router {
// JSON API.
.route("/api/isos", get(api_list_isos).post(api_upload_iso))
.route("/api/isos/:id", delete(api_delete_iso))
.route("/api/uploads", post(api_upload_begin))
.route(
"/api/uploads/:upload_id",
put(api_upload_chunk).delete(api_upload_abort),
)
// Per-ISO password prompt. PUT body `{ "password": "..." }`
// sets, `{ "password": null }` (or DELETE) clears.
.route(
@@ -87,10 +93,13 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/log/clear", post(log_stream::clear))
// Phase 4: operator terminal commands (whitelisted).
.route("/api/terminal", post(terminal::run_command))
// Phase 5: per-MAC host bindings (Tinkerbell-style). Operator
// Phase 5: per-MAC host bindings. Operator
// pins a MAC to a boot entry; /boot.ipxe?mac=... chains directly.
.route("/api/hosts", get(api_hosts_list).post(api_hosts_upsert))
.route("/api/hosts/:mac", delete(api_hosts_remove))
// Rolling "host log" of boot events: what image actually
// started installing on what MAC/IP, and when. Persisted to disk.
.route("/api/boot-log", get(api_boot_log))
// Phase 5: Prometheus scrape endpoint. Plain text exposition
// format. No auth — the metrics surface is intentionally
// boring (counts, no payloads).
@@ -173,7 +182,16 @@ fn text_plain(body: String) -> Response {
/// requesting client carries a `?mac=...` query param (iPXE's `${mac}`
/// substitution) and that MAC has a binding, we short-circuit straight
/// to the bound target instead of rendering the menu.
async fn boot_top_menu(State(state): State<AppState>, Query(p): Query<BootMenuParams>) -> Response {
async fn boot_top_menu(
State(state): State<AppState>,
peer: Option<ConnectInfo<SocketAddr>>,
Query(p): Query<BootMenuParams>,
) -> Response {
// `ConnectInfo` is only populated when axum was started with
// `into_make_service_with_connect_info` (production path). Tests
// call the router via `oneshot`, which skips that wiring — we
// tolerate it by treating the peer as unknown rather than 500ing.
let peer_ip = peer.map(|c| c.0.ip());
state
.metrics
.record_http(openpxe_core::HttpRoute::BootScript);
@@ -192,14 +210,33 @@ async fn boot_top_menu(State(state): State<AppState>, Query(p): Query<BootMenuPa
mac = %binding.mac, target = %binding.target,
"host binding applied"
);
// Pre-record the host-binding event. Reserved menu shortcuts
// (`_local`, `_queue`, …) are operator-driven non-imaging
// targets — recording them would clutter the Host log with
// routine console activity, so we skip those and only record
// for real boot-entry ids.
if !binding.target.starts_with('_') {
let title = lookup_entry_title(&isos, &binding.target);
state.boot_log.record(&BootEvent {
timestamp: time::OffsetDateTime::now_utc(),
mac: Some(binding.mac.clone()),
ip: peer_ip,
target_id: binding.target.clone(),
target_title: title,
});
}
let target = binding.target;
let bound_mac = binding.mac;
// Reserved menu shortcuts are emitted as `_xxx`; per-entry
// boot scripts are at `/boot/<id>.ipxe`. Both share the same
// `/boot/<name>` route, so the URL is identical.
// `/boot/<name>` route, so the URL is identical. We forward
// `?mac=` so the per-entry handler can record the boot into
// the Host log without depending on iPXE substitution at
// this stage.
return text_plain(format!(
"#!ipxe\n\
echo OpenPXE: per-MAC binding -> {target}\n\
chain {base}/boot/{target}.ipxe || chain {base}/boot.ipxe\n"
chain {base}/boot/{target}.ipxe?mac={bound_mac} || chain {base}/boot.ipxe\n"
));
}
}
@@ -207,6 +244,22 @@ async fn boot_top_menu(State(state): State<AppState>, Query(p): Query<BootMenuPa
text_plain(render_menu(&isos, &settings, base))
}
/// Best-effort human title for a boot entry id — falls back to the id
/// itself if the ISO has been deleted between record-time and now.
fn lookup_entry_title(isos: &[openpxe_iso_store::IsoMeta], target_id: &str) -> String {
for iso in isos {
for e in &iso.boot_entries {
if e.id == target_id {
// ISO filename plus the entry title gives the operator
// both "which image" and "which variant" (e.g. wimboot
// vs sanboot) at a glance.
return format!("{}{}", iso.filename, e.title);
}
}
}
target_id.to_string()
}
#[derive(Debug, Deserialize)]
struct BootMenuParams {
/// Client MAC, supplied by iPXE via `${mac}` variable in
@@ -222,13 +275,19 @@ struct BootSubParams {
/// encoding. Absent on the first request — that's how we know the
/// client hasn't been prompted yet.
token: Option<String>,
/// Client MAC, supplied by iPXE via `${mac}` in the chain URLs we
/// render. Optional — older bookmarks may omit it; the boot log
/// just records `None` in that case rather than refusing to boot.
mac: Option<String>,
}
async fn boot_sub(
State(state): State<AppState>,
peer: Option<ConnectInfo<SocketAddr>>,
AxumPath(filename): AxumPath<String>,
Query(p): Query<BootSubParams>,
) -> Response {
let peer_ip = peer.map(|c| c.0.ip());
// `/boot/<name>.ipxe` where `<name>` is either one of our reserved
// submenu names (prefixed `_`) or a boot entry id.
let name = filename.strip_suffix(".ipxe").unwrap_or(&filename);
@@ -300,6 +359,22 @@ async fn boot_sub(
}
}
}
// Record the boot event. This is the canonical
// moment: password gate (if any) passed, and the
// script is about to be served — i.e. the client
// is genuinely about to start imaging.
let mac_normalized = p
.mac
.as_deref()
.map(openpxe_core::normalize_mac)
.filter(|m| !m.is_empty());
state.boot_log.record(&BootEvent {
timestamp: time::OffsetDateTime::now_utc(),
mac: mac_normalized,
ip: peer_ip,
target_id: entry.id.clone(),
target_title: format!("{} — {}", iso.filename, entry.title),
});
return text_plain(render_entry(entry, &settings, base));
}
}
@@ -546,31 +621,278 @@ async fn api_clear_iso_password(
}
async fn api_upload_iso(State(state): State<AppState>, mut multipart: Multipart) -> Response {
while let Ok(Some(mut field)) = multipart.next_field().await {
if field.name() != Some("file") {
continue;
}
let filename = field.file_name().unwrap_or("uploaded.iso").to_string();
if !filename.to_ascii_lowercase().ends_with(".iso") {
return (StatusCode::BAD_REQUEST, "only .iso uploads accepted").into_response();
}
let mut handle = match state.iso_store.begin_upload(&filename).await {
Ok(h) => h,
Err(e) => return (StatusCode::CONFLICT, format!("{e}")).into_response(),
};
while let Ok(Some(chunk)) = field.chunk().await {
if let Err(e) = handle.write_chunk(&chunk).await {
let _ = handle.abort().await;
return (StatusCode::INTERNAL_SERVER_ERROR, format!("{e}")).into_response();
// Walk multipart parts until we find the file. Each branch logs so an
// operator chasing a "stuck" upload in the Terminal tab can see
// exactly which stage failed (no field, wrong field name, parser
// error, mid-stream drop, sha mismatch on finish, etc.).
loop {
let field_res = multipart.next_field().await;
match field_res {
Ok(Some(mut field)) => {
if field.name() != Some("file") {
tracing::debug!(
target: "openpxe::http::upload",
field = field.name().unwrap_or("?"),
"skipping non-file multipart part"
);
continue;
}
let filename = field.file_name().unwrap_or("uploaded.iso").to_string();
if !filename.to_ascii_lowercase().ends_with(".iso") {
tracing::warn!(
target: "openpxe::http::upload",
filename = %filename, "rejecting non-.iso upload"
);
return (StatusCode::BAD_REQUEST, "only .iso uploads accepted").into_response();
}
tracing::info!(
target: "openpxe::http::upload",
filename = %filename, "upload started"
);
let mut handle = match state.iso_store.begin_upload(&filename).await {
Ok(h) => h,
Err(e) => {
tracing::warn!(
target: "openpxe::http::upload",
filename = %filename, error = %e,
"begin_upload rejected (likely duplicate name)"
);
return (StatusCode::CONFLICT, format!("{e}")).into_response();
}
};
// Streamed reader loop. We use an explicit `match` instead
// of `while let Ok(Some(_))` so a mid-stream `Err(_)` (a
// truncated body from a reverse proxy 524 / network drop)
// is treated as a failure rather than silently completing
// with a partial file.
let mut bytes: u64 = 0;
let mut next_log_at: u64 = 64 * 1024 * 1024;
loop {
match field.chunk().await {
Ok(Some(chunk)) => {
if let Err(e) = handle.write_chunk(&chunk).await {
tracing::error!(
target: "openpxe::http::upload",
filename = %filename, bytes,
error = %e, "write_chunk failed; aborting"
);
let _ = handle.abort().await;
return (StatusCode::INTERNAL_SERVER_ERROR, format!("{e}"))
.into_response();
}
bytes += chunk.len() as u64;
if bytes >= next_log_at {
tracing::info!(
target: "openpxe::http::upload",
filename = %filename,
received_bytes = bytes,
"upload streaming"
);
// Backoff log cadence: 64 MB, 128, 256, …
next_log_at = next_log_at.saturating_mul(2);
}
}
Ok(None) => break,
Err(e) => {
tracing::error!(
target: "openpxe::http::upload",
filename = %filename, received_bytes = bytes,
error = %e,
"multipart stream ended with error (likely client \
disconnect or reverse-proxy buffer cap); aborting"
);
let _ = handle.abort().await;
return (
StatusCode::BAD_REQUEST,
format!(
"upload truncated after {bytes} bytes: {e}. \
If you went through a reverse proxy, try the \
LAN IP directly — large body buffering caps \
(Cloudflare free tier is 100 MB) commonly \
cause this."
),
)
.into_response();
}
}
}
tracing::info!(
target: "openpxe::http::upload",
filename = %filename, received_bytes = bytes,
"upload body complete; introspecting"
);
let meta = match handle.finish(&state.iso_store).await {
Ok(m) => m,
Err(e) => {
tracing::error!(
target: "openpxe::http::upload",
filename = %filename, error = %e,
"finish failed (rename/introspect)"
);
return (StatusCode::INTERNAL_SERVER_ERROR, format!("{e}")).into_response();
}
};
tracing::info!(
target: "openpxe::http::upload",
iso = %meta.id, size = meta.size_bytes,
family = ?meta.introspection.family,
entries = meta.boot_entries.len(),
"upload finished"
);
return (StatusCode::CREATED, Json(meta)).into_response();
}
Ok(None) => {
tracing::warn!(target: "openpxe::http::upload", "upload had no 'file' part");
return (StatusCode::BAD_REQUEST, "no 'file' part").into_response();
}
Err(e) => {
tracing::error!(
target: "openpxe::http::upload",
error = %e,
"multipart parser error before reading any field"
);
return (
StatusCode::BAD_REQUEST,
format!("multipart parse error: {e}"),
)
.into_response();
}
}
let meta = match handle.finish(&state.iso_store).await {
Ok(m) => m,
Err(e) => return (StatusCode::INTERNAL_SERVER_ERROR, format!("{e}")).into_response(),
};
return (StatusCode::CREATED, Json(meta)).into_response();
}
(StatusCode::BAD_REQUEST, "no 'file' part").into_response()
}
#[derive(Debug, Deserialize)]
struct UploadBeginBody {
filename: String,
#[serde(default)]
size_bytes: Option<u64>,
}
async fn api_upload_begin(
State(state): State<AppState>,
Json(body): Json<UploadBeginBody>,
) -> Response {
match state
.uploads
.begin(&state.iso_store, &body.filename, body.size_bytes)
.await
{
Ok(started) => {
tracing::info!(
target: "openpxe::http::upload",
upload_id = %started.upload_id,
iso = %started.iso_id,
filename = %started.filename,
expected_size = ?body.size_bytes,
"chunked upload started"
);
(StatusCode::CREATED, Json(started)).into_response()
}
Err(Error::Invalid(e)) if e.contains("already exists") => {
(StatusCode::CONFLICT, e).into_response()
}
Err(Error::Invalid(e)) => (StatusCode::BAD_REQUEST, e).into_response(),
Err(e) => {
tracing::error!(target: "openpxe::http::upload", error = %e, "chunked upload begin failed");
(StatusCode::INTERNAL_SERVER_ERROR, format!("{e}")).into_response()
}
}
}
async fn api_upload_chunk(
State(state): State<AppState>,
AxumPath(upload_id): AxumPath<String>,
headers: HeaderMap,
chunk: Bytes,
) -> Response {
let Some(offset) = parse_u64_header(&headers, "x-openpxe-upload-offset") else {
return (
StatusCode::BAD_REQUEST,
"missing or invalid x-openpxe-upload-offset",
)
.into_response();
};
let complete = bool_header(&headers, "x-openpxe-upload-complete");
match state
.uploads
.append(&state.iso_store, &upload_id, offset, chunk, complete)
.await
{
Ok(crate::uploads::UploadAppend::Progress { offset }) => (
StatusCode::ACCEPTED,
Json(json!({
"ok": true,
"upload_id": upload_id,
"offset": offset,
"complete": false,
})),
)
.into_response(),
Ok(crate::uploads::UploadAppend::Complete { offset, iso }) => {
tracing::info!(
target: "openpxe::http::upload",
upload_id = %upload_id,
iso = %iso.id,
size = iso.size_bytes,
family = ?iso.introspection.family,
entries = iso.boot_entries.len(),
"chunked upload finished"
);
(
StatusCode::CREATED,
Json(json!({
"ok": true,
"upload_id": upload_id,
"offset": offset,
"complete": true,
"iso": iso,
})),
)
.into_response()
}
Err(Error::Invalid(e)) if e.starts_with("expected offset") => {
(StatusCode::CONFLICT, e).into_response()
}
Err(Error::Invalid(e)) if e.starts_with("no such upload") => {
(StatusCode::NOT_FOUND, e).into_response()
}
Err(Error::Invalid(e)) => (StatusCode::BAD_REQUEST, e).into_response(),
Err(e) => {
tracing::error!(
target: "openpxe::http::upload",
upload_id = %upload_id,
error = %e,
"chunked upload failed"
);
(StatusCode::INTERNAL_SERVER_ERROR, format!("{e}")).into_response()
}
}
}
async fn api_upload_abort(
State(state): State<AppState>,
AxumPath(upload_id): AxumPath<String>,
) -> Response {
match state.uploads.abort(&upload_id).await {
Ok(()) => StatusCode::NO_CONTENT.into_response(),
Err(Error::Invalid(e)) if e.starts_with("no such upload") => {
(StatusCode::NOT_FOUND, e).into_response()
}
Err(e) => (StatusCode::INTERNAL_SERVER_ERROR, format!("{e}")).into_response(),
}
}
fn parse_u64_header(headers: &HeaderMap, name: &'static str) -> Option<u64> {
headers.get(name)?.to_str().ok()?.trim().parse::<u64>().ok()
}
fn bool_header(headers: &HeaderMap, name: &'static str) -> bool {
headers
.get(name)
.and_then(|v| v.to_str().ok())
.map(str::trim)
.is_some_and(|v| matches!(v, "1" | "true" | "TRUE" | "yes" | "YES"))
}
// ─── health / readiness ───────────────────────────────────────────────────
@@ -988,6 +1310,12 @@ async fn api_hosts_remove(
}
}
// ─── Boot event log ───────────────────────────────────────────────────────
async fn api_boot_log(State(state): State<AppState>) -> Json<serde_json::Value> {
Json(json!({ "events": state.boot_log.list() }))
}
// ─── Prometheus metrics ───────────────────────────────────────────────────
async fn api_metrics(State(state): State<AppState>) -> Response {
+14 -2
View File
@@ -182,7 +182,14 @@ pub fn render_family_menu(isos: &[IsoMeta], base_url: &str, is_windows: bool) ->
s,
"iseq ${{target}} back && chain {base}/boot.ipxe || goto menu"
);
let _ = writeln!(s, "chain {base}/boot/${{target}}.ipxe || goto menu");
// Pass `?mac=${mac}` so the per-entry handler can record the booting
// client into the Host log. iPXE substitutes `${mac}` before
// the HTTP fetch; if the firmware can't resolve it the literal
// `${mac}` is sent and the server treats it as "unknown".
let _ = writeln!(
s,
"chain {base}/boot/${{target}}.ipxe?mac=${{mac}} || goto menu"
);
s
}
@@ -461,9 +468,14 @@ pub fn render_password_prompt(entry_id: &str, iso_filename: &str, base_url: &str
);
let _ = writeln!(s, ":submit");
let _ = writeln!(s, "echo Verifying...");
// Carry `mac=${mac}` alongside the token so a successful unlock
// records the actual client MAC into the Host log. On
// older iPXE that can't resolve `${mac}` the server just stores it
// as "unknown" rather than refusing to boot.
let _ = writeln!(
s,
"chain {base}/boot/{entry_id}.ipxe?token=${{password:uristring}} || chain {base}/boot.ipxe"
"chain {base}/boot/{entry_id}.ipxe?token=${{password:uristring}}&mac=${{mac}} \
|| chain {base}/boot.ipxe"
);
s
}
+1
View File
@@ -19,6 +19,7 @@ pub mod iso_fs;
pub mod log_stream;
pub mod state;
pub mod terminal;
pub mod uploads;
pub use app::build_router;
pub use state::AppState;
+12 -1
View File
@@ -1,4 +1,7 @@
use openpxe_core::{ClientRegistry, DeploymentQueue, HostBindings, LogBus, Metrics, SettingsStore};
use crate::uploads::UploadSessions;
use openpxe_core::{
BootLog, ClientRegistry, DeploymentQueue, HostBindings, LogBus, Metrics, SettingsStore,
};
use openpxe_iso_store::{IsoStore, NfsManager, SmbManager};
use std::sync::Arc;
use time::OffsetDateTime;
@@ -13,6 +16,10 @@ pub struct AppState {
/// these MACs requests `/boot.ipxe`, we chain straight to the
/// configured target instead of rendering the menu.
pub hosts: HostBindings,
/// Persistent boot-event log surfaced under the Hosts tab. Records
/// every `/boot/<entry>.ipxe` chain that goes on to serve a script
/// (i.e. an image actually starting to install on a machine).
pub boot_log: BootLog,
/// Lock-free metrics counters surfaced at `/metrics` in Prometheus
/// text format. Cheap to clone (handles to atomics).
pub metrics: Metrics,
@@ -25,6 +32,10 @@ pub struct AppState {
/// available in the runtime image. Surfaces errors per-mount rather
/// than failing the global state.
pub nfs: NfsManager,
/// Browser chunked upload state. Multipart uploads still go straight
/// through `IsoStore`, but the UI uses sessions so large ISO transfers
/// can show deterministic progress and leave visible partial files.
pub uploads: UploadSessions,
/// Live log bus consumed by the Terminal tab via SSE. Operator-issued
/// terminal commands also push synthetic lines onto it so the tail
/// shows them inline.
+180
View File
@@ -0,0 +1,180 @@
//! Chunked upload sessions for browser-driven ISO uploads.
//!
//! The legacy multipart endpoint still exists for simple API clients, but
//! browsers get a better failure mode with raw chunks: progress advances after
//! each acknowledged write, partial files appear in the ISO directory
//! immediately, and reverse proxies are less likely to buffer an entire DVD
//! image before OpenPXE sees byte one.
use bytes::Bytes;
use openpxe_core::{Error, Result};
use openpxe_iso_store::{IsoMeta, IsoStore, UploadHandle};
use serde::Serialize;
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::Mutex;
use uuid::Uuid;
const DEFAULT_CHUNK_SIZE: u64 = 8 * 1024 * 1024;
#[derive(Clone, Default)]
pub struct UploadSessions {
inner: Arc<Mutex<HashMap<String, Arc<Mutex<UploadSession>>>>>,
}
struct UploadSession {
filename: String,
expected_size: Option<u64>,
offset: u64,
handle: Option<UploadHandle>,
}
#[derive(Debug, Clone, Serialize)]
pub struct UploadStarted {
pub upload_id: String,
pub iso_id: String,
pub filename: String,
pub offset: u64,
pub chunk_size: u64,
}
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum UploadAppend {
Progress { offset: u64 },
Complete { offset: u64, iso: Box<IsoMeta> },
}
impl UploadSessions {
pub async fn begin(
&self,
store: &IsoStore,
filename: &str,
expected_size: Option<u64>,
) -> Result<UploadStarted> {
if !filename.to_ascii_lowercase().ends_with(".iso") {
return Err(Error::Invalid("only .iso uploads accepted".to_string()));
}
let handle = store.begin_upload(filename).await?;
let iso_id = handle.id.clone();
let upload_id = Uuid::new_v4().to_string();
let session = UploadSession {
filename: filename.to_string(),
expected_size,
offset: 0,
handle: Some(handle),
};
self.inner
.lock()
.await
.insert(upload_id.clone(), Arc::new(Mutex::new(session)));
Ok(UploadStarted {
upload_id,
iso_id,
filename: filename.to_string(),
offset: 0,
chunk_size: DEFAULT_CHUNK_SIZE,
})
}
pub async fn append(
&self,
store: &IsoStore,
upload_id: &str,
offset: u64,
chunk: Bytes,
complete: bool,
) -> Result<UploadAppend> {
let Some(session_lock) = self.inner.lock().await.get(upload_id).cloned() else {
return Err(Error::Invalid(format!("no such upload '{upload_id}'")));
};
let mut session = session_lock.lock().await;
if session.offset != offset {
return Err(Error::Invalid(format!(
"expected offset {}, got {offset}",
session.offset
)));
}
let new_offset = session
.offset
.checked_add(chunk.len() as u64)
.ok_or_else(|| Error::Invalid("upload offset overflow".to_string()))?;
if let Some(expected) = session.expected_size {
if new_offset > expected {
return Err(Error::Invalid(format!(
"chunk exceeds declared upload size {expected}"
)));
}
}
let Some(handle) = session.handle.as_mut() else {
return Err(Error::Invalid("upload already completed".to_string()));
};
if let Err(e) = handle.write_chunk(&chunk).await {
let handle = session.handle.take();
drop(session);
self.inner.lock().await.remove(upload_id);
if let Some(handle) = handle {
let _ = handle.abort().await;
}
return Err(e);
}
session.offset = new_offset;
if !complete {
return Ok(UploadAppend::Progress { offset: new_offset });
}
if let Some(expected) = session.expected_size {
if new_offset != expected {
return Err(Error::Invalid(format!(
"final chunk ended at {new_offset}, expected {expected}"
)));
}
}
let Some(handle) = session.handle.take() else {
return Err(Error::Invalid("upload already completed".to_string()));
};
let filename = session.filename.clone();
drop(session);
tracing::info!(
target: "openpxe::http::upload",
upload_id,
filename = %filename,
received_bytes = new_offset,
"chunked upload body complete; introspecting"
);
let meta = match handle.finish(store).await {
Ok(meta) => meta,
Err(e) => {
self.inner.lock().await.remove(upload_id);
return Err(e);
}
};
self.inner.lock().await.remove(upload_id);
Ok(UploadAppend::Complete {
offset: new_offset,
iso: Box::new(meta),
})
}
pub async fn abort(&self, upload_id: &str) -> Result<()> {
let Some(session_lock) = self.inner.lock().await.remove(upload_id) else {
return Err(Error::Invalid(format!("no such upload '{upload_id}'")));
};
let mut session = session_lock.lock().await;
if let Some(handle) = session.handle.take() {
handle.abort().await?;
}
Ok(())
}
}
+226
View File
@@ -98,6 +98,7 @@ async fn build_state() -> (AppState, tempfile::TempDir) {
iso_store.set_nfs_root(nfs.mount_root());
let log_bus = LogBus::new(64);
let hosts = HostBindings::load_or_default(dir.path());
let boot_log = openpxe_core::BootLog::load_or_default(dir.path());
let metrics = Metrics::new();
let state = AppState {
iso_store,
@@ -105,9 +106,11 @@ async fn build_state() -> (AppState, tempfile::TempDir) {
queue,
settings,
hosts,
boot_log,
metrics,
smb: None,
nfs,
uploads: openpxe_http_api::uploads::UploadSessions::default(),
log_bus,
started_at: time::OffsetDateTime::now_utc(),
public_base_url: "http://127.0.0.1".into(),
@@ -975,3 +978,226 @@ async fn set_password_for_unknown_iso_returns_404() {
.unwrap();
assert_eq!(res.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn boot_log_records_entry_serve_with_mac() {
// End-to-end: upload an ISO, fetch the entry's boot script with a
// MAC query param, then GET /api/boot-log and assert the event is
// there with the supplied mac.
let (state, _dir) = build_state().await;
let app = build_router(state);
let (ct, body) = multipart_iso_body("fake-alpine.iso", &fake_alpine_iso());
let upload = app
.clone()
.oneshot(
Request::builder()
.method("POST")
.uri("/api/isos")
.header("content-type", ct)
.body(Body::from(body))
.unwrap(),
)
.await
.unwrap();
assert_eq!(upload.status(), StatusCode::CREATED);
// Fetch the per-entry script with ?mac=...
let (s, _) = get(&app, "/boot/fake-alpine-linux.ipxe?mac=AA:BB:CC:00:00:09").await;
assert_eq!(s, StatusCode::OK);
// The boot log should now contain exactly one entry, with the
// normalized MAC and our target id.
let (s, body) = get(&app, "/api/boot-log").await;
assert_eq!(s, StatusCode::OK);
let v: serde_json::Value = serde_json::from_slice(&body).unwrap();
let events = v["events"].as_array().expect("events");
assert_eq!(events.len(), 1);
let ev = &events[0];
assert_eq!(ev["target_id"], "fake-alpine-linux");
assert_eq!(ev["mac"], "aa:bb:cc:00:00:09"); // normalized
// Title should include the filename and entry title.
let title = ev["target_title"].as_str().unwrap();
assert!(title.contains("fake-alpine.iso"), "title was {title}");
}
#[tokio::test]
async fn boot_log_endpoint_empty_when_no_boots() {
let (state, _dir) = build_state().await;
let app = build_router(state);
let (s, body) = get(&app, "/api/boot-log").await;
assert_eq!(s, StatusCode::OK);
let v: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert!(v["events"].as_array().unwrap().is_empty());
}
#[tokio::test]
async fn boot_log_does_not_record_reserved_menu_targets() {
// Reserved targets (_local, _queue, …) are operator console actions,
// not imaging events. The Hosts log skips them so it stays focused
// on "what got installed where".
let (state, _dir) = build_state().await;
let app = build_router(state.clone());
// Bind a MAC to the _local shortcut and hit /boot.ipxe.
let body = r#"{"mac":"aa:bb:cc:00:00:11","target":"_local","label":"q"}"#;
let (s, _) = post_json(&app, "/api/hosts", body).await;
assert_eq!(s, StatusCode::CREATED);
let (s, _) = get(&app, "/boot.ipxe?mac=aa:bb:cc:00:00:11").await;
assert_eq!(s, StatusCode::OK);
let (_, body) = get(&app, "/api/boot-log").await;
let v: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert!(
v["events"].as_array().unwrap().is_empty(),
"reserved targets should not appear in boot log; got {v}"
);
}
#[tokio::test]
async fn upload_rejects_non_iso_filename_with_clear_message() {
// Sanity for the upload-logging path: a wrong extension should land
// a 400 with the human message rather than silently being eaten by
// the multipart loop. (No iso ends up in the store either.)
let (state, _dir) = build_state().await;
let app = build_router(state);
let (ct, body) = multipart_iso_body("not-an-iso.txt", b"hello world");
let res = app
.oneshot(
Request::builder()
.method("POST")
.uri("/api/isos")
.header("content-type", ct)
.body(Body::from(body))
.unwrap(),
)
.await
.unwrap();
assert_eq!(res.status(), StatusCode::BAD_REQUEST);
let body = axum::body::to_bytes(res.into_body(), usize::MAX)
.await
.unwrap();
let text = std::str::from_utf8(&body).unwrap();
assert!(text.contains("only .iso uploads accepted"), "got: {text}");
}
#[tokio::test]
async fn chunked_upload_writes_progressively_and_finishes_iso() {
let (state, dir) = build_state().await;
let app = build_router(state);
let iso = fake_alpine_iso();
let start = app
.clone()
.oneshot(
Request::builder()
.method("POST")
.uri("/api/uploads")
.header("content-type", "application/json")
.body(Body::from(
r#"{"filename":"chunked-alpine.iso","size_bytes":65536}"#,
))
.unwrap(),
)
.await
.unwrap();
assert_eq!(start.status(), StatusCode::CREATED);
let body = axum::body::to_bytes(start.into_body(), usize::MAX)
.await
.unwrap();
let started: serde_json::Value = serde_json::from_slice(&body).unwrap();
let upload_id = started["upload_id"].as_str().unwrap();
let split = 8192usize;
let first = app
.clone()
.oneshot(
Request::builder()
.method("PUT")
.uri(format!("/api/uploads/{upload_id}"))
.header("x-openpxe-upload-offset", "0")
.body(Body::from(iso[..split].to_vec()))
.unwrap(),
)
.await
.unwrap();
assert_eq!(first.status(), StatusCode::ACCEPTED);
let body = axum::body::to_bytes(first.into_body(), usize::MAX)
.await
.unwrap();
let progress: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert_eq!(progress["offset"].as_u64().unwrap(), split as u64);
assert!(!progress["complete"].as_bool().unwrap());
assert!(
dir.path().join("isos/chunked-alpine.partial").exists(),
"chunked upload should leave a visible partial file while in progress"
);
let final_chunk = app
.clone()
.oneshot(
Request::builder()
.method("PUT")
.uri(format!("/api/uploads/{upload_id}"))
.header("x-openpxe-upload-offset", split.to_string())
.header("x-openpxe-upload-complete", "true")
.body(Body::from(iso[split..].to_vec()))
.unwrap(),
)
.await
.unwrap();
assert_eq!(final_chunk.status(), StatusCode::CREATED);
let body = axum::body::to_bytes(final_chunk.into_body(), usize::MAX)
.await
.unwrap();
let finished: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert!(finished["complete"].as_bool().unwrap());
assert_eq!(finished["iso"]["id"], "chunked-alpine");
assert!(dir.path().join("isos/chunked-alpine.iso").exists());
assert!(!dir.path().join("isos/chunked-alpine.partial").exists());
}
#[tokio::test]
async fn chunked_upload_rejects_offset_mismatch_without_advancing() {
let (state, _dir) = build_state().await;
let app = build_router(state);
let start = app
.clone()
.oneshot(
Request::builder()
.method("POST")
.uri("/api/uploads")
.header("content-type", "application/json")
.body(Body::from(
r#"{"filename":"offset-test.iso","size_bytes":16}"#,
))
.unwrap(),
)
.await
.unwrap();
assert_eq!(start.status(), StatusCode::CREATED);
let body = axum::body::to_bytes(start.into_body(), usize::MAX)
.await
.unwrap();
let started: serde_json::Value = serde_json::from_slice(&body).unwrap();
let upload_id = started["upload_id"].as_str().unwrap();
let mismatch = app
.oneshot(
Request::builder()
.method("PUT")
.uri(format!("/api/uploads/{upload_id}"))
.header("x-openpxe-upload-offset", "8")
.body(Body::from(vec![1, 2, 3, 4]))
.unwrap(),
)
.await
.unwrap();
assert_eq!(mismatch.status(), StatusCode::CONFLICT);
let body = axum::body::to_bytes(mismatch.into_body(), usize::MAX)
.await
.unwrap();
let text = String::from_utf8(body.to_vec()).unwrap();
assert!(text.contains("expected offset 0"), "got: {text}");
}
+16
View File
@@ -216,6 +216,9 @@ impl IsoStore {
return Err(Error::Invalid(format!("iso '{id}' already exists")));
}
let partial_path = self.iso_dir.join(format!("{id}.partial"));
if partial_path.exists() {
return Err(Error::Invalid(format!("iso '{id}' is already uploading")));
}
let file = tokio::fs::File::create(&partial_path).await?;
Ok(UploadHandle {
id,
@@ -594,6 +597,19 @@ mod tests {
assert!(matches!(r, Err(Error::Invalid(_))));
}
#[tokio::test]
async fn begin_upload_rejects_existing_partial_file() {
let dir = tempdir().unwrap();
let store = IsoStore::new(dir.path().to_path_buf());
store.ensure_dirs().await.unwrap();
tokio::fs::write(dir.path().join("ubuntu.partial"), b"in-flight")
.await
.unwrap();
let r = store.begin_upload("ubuntu.iso").await;
assert!(matches!(r, Err(Error::Invalid(_))));
}
#[tokio::test]
async fn password_persists_via_meta_json_for_local_isos() {
// Hash makes it onto disk so it survives a restart.
+13 -2
View File
@@ -39,7 +39,7 @@ enum Command {
/// docker run --rm \
/// -v /my/isos:/seed:ro \
/// -v openpxe-data:/var/lib/openpxe/isos \
/// openpxe:0.3.2 seed --from /seed
/// openpxe:0.4.1 seed --from /seed
Seed {
/// Source directory containing one or more `.iso` files.
#[arg(long)]
@@ -103,6 +103,7 @@ async fn main() -> anyhow::Result<()> {
let queue = DeploymentQueue::new();
let settings = SettingsStore::load_or_default(&config.paths.work_dir);
let hosts = HostBindings::load_or_default(&config.paths.work_dir);
let boot_log = openpxe_core::BootLog::load_or_default(&config.paths.work_dir);
let metrics = Metrics::new();
// Build the SMB manager unconditionally — it starts/stops on the
@@ -140,9 +141,11 @@ async fn main() -> anyhow::Result<()> {
settings: settings.clone(),
queue: queue.clone(),
hosts: hosts.clone(),
boot_log: boot_log.clone(),
metrics: metrics.clone(),
smb: Some(smb.clone()),
nfs: nfs.clone(),
uploads: openpxe_http_api::uploads::UploadSessions::default(),
log_bus: log_bus.clone(),
started_at: time::OffsetDateTime::now_utc(),
public_base_url: public_base_url.clone(),
@@ -156,7 +159,15 @@ async fn main() -> anyhow::Result<()> {
let http_task = tokio::spawn(async move {
let listener = tokio::net::TcpListener::bind(http_addr).await?;
tracing::info!(target: "openpxe::http", "HTTP listening on {http_addr}");
axum::serve(listener, router).await?;
// `into_make_service_with_connect_info` is required so per-request
// `ConnectInfo<SocketAddr>` extractors can resolve the peer IP —
// used by `/boot/<entry>.ipxe` to record the booting client's
// address into the Host log. Without this the extractor 500s.
axum::serve(
listener,
router.into_make_service_with_connect_info::<std::net::SocketAddr>(),
)
.await?;
Ok::<_, anyhow::Error>(())
});
+48 -25
View File
@@ -8,23 +8,26 @@
* CSS lands). */
:root {
/* Dark palette (default). */
--bg: #0b1018;
--bg-panel: #121826;
--bg-panel-2: #1a2334;
--bg-elev: #223047;
--fg: #e4e8ef;
--fg-dim: #8a94a7;
--fg-dimmer: #5a6379;
--accent: #00d4b4; /* Netbox-ish teal */
/* Jet-black dark palette (default). Modelled on Netbox Labs's
near-black product chrome, with surfaces stepping subtly upward
rather than the previous blue-tinted ramp, so the UI reads as a
genuine "dark" rather than "dim navy". */
--bg: #030303;
--bg-panel: #0a0a0a;
--bg-panel-2: #141414;
--bg-elev: #1c1c1c;
--fg: #e8eaed;
--fg-dim: #9aa0a6;
--fg-dimmer: #6b7077;
--accent: #00d4b4; /* Netbox-ish teal — kept for brand */
--accent-dim: #07a38c;
--warn: #ffb347;
--err: #ef6e6e;
--ok: #4ade80;
--border: #223047;
--border-soft: #172033;
--terminal-bg: #06090e;
--shadow-card: 0 1px 0 rgba(255,255,255,0.02), 0 8px 24px rgba(0,0,0,0.25);
--border: #1f1f1f;
--border-soft: #141414;
--terminal-bg: #050505;
--shadow-card: 0 1px 0 rgba(255,255,255,0.02), 0 8px 24px rgba(0,0,0,0.55);
--radius: 6px;
--radius-lg: 10px;
--sidebar-w: 240px;
@@ -38,7 +41,7 @@
consistency. Designed against Netbox Labs's reference screenshot:
near-white surfaces, soft grey dividers, dark text. */
--bg: #f6f8fb;
--bg-panel: #ffffff;
--bg-panel: #fbfcfe;
--bg-panel-2: #f0f3f8;
--bg-elev: #e6ebf2;
--fg: #1c2330;
@@ -113,10 +116,30 @@ code, kbd { font-family: var(--mono); font-size: 12.5px;
}
.sidebar nav a.active .count { background: var(--accent); color: #002923; }
.sidebar .footer {
padding: 10px 18px; border-top: 1px solid var(--border);
padding: 12px 18px; border-top: 1px solid var(--border);
color: var(--fg-dimmer); font-size: 11px;
display: flex; flex-direction: column; gap: 4px;
}
.sidebar .footer code { background: transparent; color: var(--fg-dim); padding: 0; }
.sidebar .footer code { background: transparent; color: var(--fg-dim); padding: 0;
font-size: 11px; word-break: break-all; }
.sidebar .footer .status-row {
display: flex; align-items: center; gap: 8px;
margin-bottom: 4px;
}
.sidebar .footer .status-row .dot {
width: 8px; height: 8px; border-radius: 50%; display: inline-block;
background: var(--fg-dimmer); flex: none;
}
.sidebar .footer .status-row .dot.ok { background: var(--ok);
box-shadow: 0 0 6px color-mix(in srgb, var(--ok) 60%, transparent); }
.sidebar .footer .status-row .dot.err { background: var(--err); }
.sidebar .footer .status-row .dot.warn { background: var(--warn); }
.sidebar .footer .status-label { color: var(--fg-dim); }
.sidebar .footer .status-value { color: var(--fg); font-weight: 600; }
.sidebar .footer .status-value.ok { color: var(--ok); }
.sidebar .footer .status-value.err { color: var(--err); }
.sidebar .footer .status-value.warn { color: var(--warn); }
.sidebar .footer .footer-sub { color: var(--fg-dimmer); margin-top: 2px; }
/* ── Top bar ───────────────────────────────────────────────────────── */
@@ -230,7 +253,7 @@ button, .btn {
cursor: pointer;
transition: background 0.12s ease;
}
button:hover, .btn:hover { background: var(--accent-dim); color: #fff; }
button:hover, .btn:hover { background: var(--accent-dim); color: #f4fffd; }
button.ghost { background: transparent; color: var(--fg); border: 1px solid var(--border); }
button.ghost:hover { background: var(--bg-panel-2); color: var(--fg); }
button.danger { background: transparent; color: var(--err); border: 1px solid color-mix(in srgb, var(--err) 35%, transparent); }
@@ -431,29 +454,29 @@ tr.unbootable td:first-child { border-left: 3px solid var(--warn); }
.terminal .input-row {
display: flex; align-items: center; gap: 8px;
padding: 8px 14px;
background: #0a0e15;
border-top: 1px solid #1d2330;
background: #050505;
border-top: 1px solid #181818;
}
.terminal .input-row .prompt { color: var(--accent); font-family: var(--mono); }
.terminal .input-row input {
flex: 1; background: transparent; border: 0; color: #e4e8ef;
flex: 1; background: transparent; border: 0; color: var(--fg);
font: inherit; font-family: var(--mono); font-size: 13px;
outline: none; padding: 4px 0;
}
.terminal .toolbar {
display: flex; gap: 8px; align-items: center;
padding: 8px 14px;
background: #0a0e15;
border-bottom: 1px solid #1d2330;
font-size: 12px; color: #8a94a7;
background: #050505;
border-bottom: 1px solid #181818;
font-size: 12px; color: var(--fg-dim);
}
.terminal .toolbar .right { margin-left: auto; display: flex; gap: 6px; }
.terminal .toolbar button {
padding: 3px 9px; font-size: 11px;
background: transparent; color: #8a94a7; border: 1px solid #1d2330;
background: transparent; color: var(--fg-dim); border: 1px solid #181818;
font-weight: 500;
}
.terminal .toolbar button:hover { color: #e4e8ef; background: #1d2330; }
.terminal .toolbar button:hover { color: var(--fg); background: #181818; }
/* ── About card ─────────────────────────────────────────────────── */
.about-hero { padding: 20px 24px; }
+132 -32
View File
@@ -295,7 +295,7 @@
return el('div', {class:'grid'}, [
el('div', {class:'card'}, [
el('header', {}, el('h2', {}, 'Forge')),
el('header', {}, el('h2', {}, 'Status')),
queueProgressWidget(imaging, entries.length),
]),
el('div', {class:'card'}, [
@@ -346,29 +346,84 @@
});
file.onchange = () => { if (file.files[0]) upload(file.files[0]); };
function upload(f) {
upMsg.textContent = 'Uploading ' + f.name + ' (' + fmtBytes(f.size) + ')…';
upMsg.className = 'msg';
// Chunked upload telemetry. The old browser path posted one huge
// multipart body, which left operators staring at 0% when a reverse
// proxy buffered or rejected the request before OpenPXE saw it. This
// path writes small raw chunks; each acknowledged chunk advances the
// bar and leaves a visible .partial file in the ISO directory.
async function upload(f) {
const started = Date.now();
const bar = $('#bar');
const setStatus = (text, cls) => { upMsg.textContent = text; upMsg.className = 'msg ' + (cls || ''); };
const update = (loaded, total, phase) => {
const pct = total > 0 ? Math.min(100, (loaded / total) * 100) : 100;
bar.style.width = pct.toFixed(1) + '%';
const elapsed = Math.max(0.001, (Date.now() - started) / 1000);
const rate = loaded > 0 ? loaded / elapsed : 0;
const remain = rate > 0 ? (total - loaded) / rate : 0;
setStatus(
phase + ' ' + f.name + ' - ' +
fmtBytes(loaded) + ' of ' + fmtBytes(total) +
' (' + pct.toFixed(1) + '%, ' + fmtBytes(rate) + '/s' +
(remain > 0 ? ', ' + Math.ceil(remain) + 's left' : '') + ')');
};
const failText = async (r) => {
const text = (await r.text()).slice(0, 240);
let hint = '';
if (r.status === 413) hint = ' - body too large. A proxy likely rejected this chunk.';
else if (r.status === 502) hint = ' - bad gateway. Proxy lost the upstream mid-stream.';
else if (r.status === 504) hint = ' - gateway timeout. Try the LAN IP directly.';
else if (r.status === 409) hint = ' - name conflict or offset mismatch. Remove the old ISO and retry.';
return 'HTTP ' + r.status + ' ' + text + hint;
};
let uploadId = null;
setStatus('Preparing upload for ' + f.name + ' (' + fmtBytes(f.size) + ')');
prog.classList.add('active');
const fd = new FormData(); fd.append('file', f);
const xhr = new XMLHttpRequest();
xhr.upload.onprogress = e => {
if (e.lengthComputable) $('#bar').style.width = (e.loaded/e.total*100).toFixed(1) + '%';
};
xhr.onload = () => {
prog.classList.remove('active');
$('#bar').style.width = '0';
if (xhr.status >= 200 && xhr.status < 300) {
upMsg.textContent = 'Uploaded & analyzed.'; upMsg.className = 'msg ok';
render('storage');
} else {
upMsg.textContent = 'Upload failed: ' + xhr.status + ' ' + xhr.responseText;
upMsg.className = 'msg err';
bar.style.width = '1%';
try {
const begin = await postJSON('/api/uploads', {
filename: f.name,
size_bytes: f.size,
});
if (!begin.ok) throw new Error(await failText(begin));
const session = await begin.json();
uploadId = session.upload_id;
const chunkSize = Math.max(1024 * 1024, Number(session.chunk_size || 8 * 1024 * 1024));
let offset = Number(session.offset || 0);
let finished = null;
do {
const end = Math.min(offset + chunkSize, f.size);
const complete = end >= f.size;
const r = await fetch('/api/uploads/' + encodeURIComponent(uploadId), {
method: 'PUT',
headers: {
'x-openpxe-upload-offset': String(offset),
'x-openpxe-upload-complete': complete ? 'true' : 'false',
},
body: f.slice(offset, end),
});
if (!r.ok) throw new Error(await failText(r));
const j = await r.json();
offset = Number(j.offset || end);
update(offset, f.size, complete ? 'Analyzing' : 'Uploading');
if (j.complete) finished = j.iso || true;
} while (!finished);
setStatus('Uploaded and analyzed: ' + f.name + ' (' + fmtBytes(f.size) + ')', 'ok');
render('storage');
} catch (err) {
if (uploadId) {
try { await fetch('/api/uploads/' + encodeURIComponent(uploadId), {method: 'DELETE'}); }
catch {}
}
};
xhr.onerror = () => { upMsg.textContent = 'Network error.'; upMsg.className = 'msg err'; };
xhr.open('POST', '/api/isos');
xhr.send(fd);
setStatus('Upload failed: ' + (err && err.message ? err.message : String(err)), 'err');
} finally {
prog.classList.remove('active');
if (!upMsg.className.includes('ok')) bar.style.width = '0';
}
}
// ── ISO table (mixed local + NFS) ──
@@ -602,9 +657,11 @@
},
hosts: async () => {
const [{ hosts = [] }, isos] = await Promise.all([
const [{ hosts = [] }, isos, bootLogRes] = await Promise.all([
getJSON('/api/hosts'), getJSON('/api/isos'),
getJSON('/api/boot-log').catch(() => ({ events: [] })),
]);
const bootEvents = bootLogRes.events || [];
const targets = isos.flatMap(i => i.boot_entries.map(e => ({
id: e.id, title: e.title + ' — ' + familyLabel(i.introspection.family),
})));
@@ -680,8 +737,7 @@
upsertBtn, msg,
el('p', {class:'msg', style:'margin-top:14px'},
'When a client with a bound MAC requests boot.ipxe, OpenPXE ' +
'short-circuits past the interactive menu and chains directly. ' +
'Inspired by Tinkerbell smee\'s MAC-prepended URL pattern.'),
'short-circuits past the interactive menu and chains directly.'),
]),
]),
el('div', {class:'card'}, [
@@ -691,6 +747,37 @@
]),
table,
]),
el('div', {class:'card'}, [
el('header', {}, [
el('h2', {}, 'Host log'),
el('span', {class:'sub'},
bootEvents.length + ' event' + (bootEvents.length === 1 ? '' : 's')),
]),
bootEvents.length
? el('table', {}, [
el('thead', {}, el('tr', {}, [
el('th', {}, 'Time'),
el('th', {}, 'MAC'),
el('th', {}, 'IP'),
el('th', {}, 'Image'),
])),
el('tbody', {},
bootEvents.map(e => el('tr', {}, [
el('td', {}, fmtAgo(e.timestamp)),
el('td', {class:'mono'}, e.mac || el('span', {class:'tag'}, '(unknown)')),
el('td', {class:'mono'}, e.ip ? String(e.ip) : '—'),
el('td', {}, [
el('span', {style:'font-weight:600'}, e.target_title || e.target_id),
el('div', {class:'meta',
style:'color:var(--fg-dim);font-size:11.5px;margin-top:2px'},
e.target_id),
]),
]))),
])
: el('div', {class:'empty'},
'No boot events yet. When a PXE client chains a boot entry, ' +
'it lands here with the MAC, IP, and image it received.'),
]),
]);
},
@@ -915,6 +1002,24 @@
}
}
// Set the sidebar footer "Service status:" line. The chip itself moved
// off the topbar in v0.4.x: operators wanted readiness, advertised
// URL, and the boot IP grouped together as the bottom-left summary.
function setReady(state) {
const dot = $('[data-bind=ready_dot]');
const lbl = $('[data-bind=ready_label]');
if (!dot || !lbl) return;
const map = {
ready: { cls: 'ok', text: 'Ready' },
notready: { cls: 'err', text: 'Not ready' },
unreachable: { cls: 'err', text: 'Unreachable' },
};
const m = map[state] || { cls: 'warn', text: 'Checking…' };
dot.className = 'dot ' + m.cls;
lbl.className = 'status-value ' + m.cls;
lbl.textContent = m.text;
}
async function refreshChips() {
try {
const s = await getJSON('/api/status');
@@ -924,14 +1029,9 @@
$$('[data-bind=client_count],[data-bind=client_count2]').forEach(n => n.textContent = String(s.client_count));
$$('[data-bind=queue_count],[data-bind=queue_count2]').forEach(n => n.textContent = String(s.queue_count));
$$('[data-bind=host_count]').forEach(n => n.textContent = String(s.host_bindings || 0));
const chip = $('[data-bind=ready_chip]');
if (chip) {
if (r.ok) { chip.textContent = '● ready'; chip.className = 'chip ready'; }
else { chip.textContent = '● not ready'; chip.className = 'chip notready'; }
}
setReady(r.ok ? 'ready' : 'notready');
} catch {
const chip = $('[data-bind=ready_chip]');
if (chip) { chip.textContent = '● unreachable'; chip.className = 'chip notready'; }
setReady('unreachable');
}
}
+7 -3
View File
@@ -29,7 +29,7 @@
<img src="/assets/logo.svg" alt="" />
<div>
<strong>OpenPXE</strong>
<div class="sub">v<span data-bind="version">0.3.2</span></div>
<div class="sub">v<span data-bind="version">0.4.1</span></div>
</div>
</div>
<nav>
@@ -51,7 +51,12 @@
<a data-view="about">About</a>
</nav>
<div class="footer">
Advertised to clients<br/>
<div class="status-row">
<span class="dot" data-bind="ready_dot" title="Server readiness"></span>
<span class="status-label">Service status:</span>
<span class="status-value" data-bind="ready_label">checking…</span>
</div>
<div class="footer-sub">Advertised to clients</div>
<code>{{BASE_URL}}</code>
</div>
</aside>
@@ -59,7 +64,6 @@
<header class="topbar">
<h1 data-bind="view_title">Dashboard</h1>
<div class="spacer"></div>
<span class="chip" data-bind="ready_chip" title="Server readiness">checking…</span>
<span class="chip"><strong data-bind="iso_count2">0</strong>&nbsp;images</span>
<span class="chip"><strong data-bind="client_count2">0</strong>&nbsp;clients</span>
<span class="chip"><strong data-bind="queue_count2">0</strong>&nbsp;in queue</span>
+7 -2
View File
@@ -14,7 +14,7 @@
# Debian slim at ~75 MB + binary ~25 MB is fine for a PXE server that
# spends most of its life idle.
ARG RUST_VERSION=1.82
ARG RUST_VERSION=1.95
########## fetch iPXE binaries ##########
FROM debian:12-slim AS fetch
@@ -33,7 +33,12 @@ WORKDIR /src
# to silently serve stale stub binaries when cargo's fingerprint didn't
# notice the source swap. A single build is ~1.5 min longer on cold cache
# but guarantees the binary reflects the sources we copied.
COPY Cargo.toml rust-toolchain.toml ./
# Do not copy rust-toolchain.toml into the image. The local workspace pins
# developer tooling, but inside Docker we intentionally use the Rust version
# selected by the base image. Copying rust-toolchain.toml with
# `channel = "stable"` makes rustup download a second full toolchain during
# `cargo build`, which is slow and can exhaust small Colima/CI disks.
COPY Cargo.toml Cargo.lock ./
COPY crates/ crates/
COPY --from=fetch /src/assets/ipxe /src/assets/ipxe
+1 -1
View File
@@ -34,7 +34,7 @@ spec:
fsGroup: 10001
containers:
- name: openpxe
image: gitea.milesward.dev/mward4/openpxe:0.3.2
image: gitea.milesward.dev/mward4/openpxe:0.4.1
imagePullPolicy: IfNotPresent
ports:
- name: dhcp
+7 -7
View File
@@ -6,7 +6,7 @@ boot from OpenPXE". Pick the one that matches what you have.
## Path A — build on Unraid, push to Gitea registry, pull by tag
Recommended once you've done it once. Image is published to
`gitea.milesward.dev/mward4/openpxe:0.3.2` (or your equivalent) and
`gitea.milesward.dev/mward4/openpxe:0.4.1` (or your equivalent) and
every Unraid template / docker-compose just references the tag.
Pre-flight:
@@ -40,14 +40,14 @@ What it does:
3. `docker build` against `deploy/docker/Dockerfile`.
4. `docker login gitea.milesward.dev:3000` using a temp `DOCKER_CONFIG`
so the credential never lands in your real `~/.docker/config.json`.
5. `docker push` both `:0.3.2` and `:latest`.
5. `docker push` both `:0.4.1` and `:latest`.
6. Logout, scrub the temp config, delete the workspace.
After it finishes, in Unraid → Docker → Add Container, set:
| Field | Value |
|------------|-------------------------------------------------|
| Repository | `gitea.milesward.dev/mward4/openpxe:0.3.2` |
| Repository | `gitea.milesward.dev/mward4/openpxe:0.4.1` |
| Network | `host` |
| Extra args | `--cap-add=NET_BIND_SERVICE` |
@@ -88,14 +88,14 @@ then:
```bash
# On the build host
docker save openpxe:0.3.2 | gzip > openpxe-0.3.2.tar.gz
docker save openpxe:0.4.1 | gzip > openpxe-0.4.1.tar.gz
# Transfer (rsync / scp / SMB / ZFS-replicate / sneakernet)
scp openpxe-0.3.2.tar.gz root@unraid:/tmp/
scp openpxe-0.4.1.tar.gz root@unraid:/tmp/
# On Unraid
gunzip -c /tmp/openpxe-0.3.2.tar.gz | docker load
docker tag openpxe:0.3.2 gitea.milesward.dev/mward4/openpxe:0.3.2
gunzip -c /tmp/openpxe-0.4.1.tar.gz | docker load
docker tag openpxe:0.4.1 gitea.milesward.dev/mward4/openpxe:0.4.1
```
If you want it pullable by tag from other Unraid templates, push to
+1 -1
View File
@@ -1,6 +1,6 @@
# Phase 6 — recommendations
The v0.3.2 cut leaves OpenPXE in a state where the entire protocol stack
The v0.4.1 cut leaves OpenPXE in a state where the entire protocol stack
and operator UI are exercised by the automated test suite, the container is
multi-arch buildable, and the image ships at ~97 MB. What's left before
this looks and feels like a 1.0 product is mostly **real-hardware
+3 -4
View File
@@ -265,10 +265,9 @@ tab is one click from the brand bar.
- Persisted to `<work_dir>/hosts.json`. Like `SettingsStore`, in-memory
is authoritative — disk corruption falls back to empty rather than
failing startup.
- Inspired by Tinkerbell `smee`'s MAC-prepended URL pattern. The DHCP
reply now embeds `?mac=${mac}` in the boot.ipxe URL; iPXE substitutes
the literal MAC client-side, so the HTTP layer can short-circuit
past the menu when a binding exists.
- The DHCP reply embeds `?mac=${mac}` in the boot.ipxe URL; iPXE
substitutes the literal MAC client-side, so the HTTP layer can
short-circuit past the menu when a binding exists.
- `/api/hosts` GET / POST / DELETE drives the **Hosts** tab.
**Prometheus metrics** (`crates/core/src/metrics.rs`):