mward4/OpenPXE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 40 Wiki Activity

Initial commit: PXEForge Phases 1-4

Browse Source 
Container-native PXE boot server in Rust, designed as a clean-room
alternative to iVentoy that never touches the client OS trust store.
This is the first commit of the project; it lands the full output of
Phases 1, 2, 3, and 4 in one shot.

## Phase 1 — protocol stack

- 8-crate workspace (core, dhcp-proxy, tftp, http-api, iso-store,
  ipxe-assets, webui, pxeforge bin).
- DHCP proxy (RFC 4578): replies with boot info only, never leases —
  sidesteps CAP_NET_RAW. Architecture-aware bootfile selection from
  option 93 (BIOS, IA32, x64-UEFI alias 0x0007/0x0009, ARM64).
- TFTP server with full OACK negotiation: blksize, tsize, windowsize.
  Without it a 1 MiB iPXE binary takes 2000 packets and unusably long.
- Two-stage iPXE chain: firmware PXE -> TFTP iPXE binary -> iPXE
  re-DHCPs with user-class iPXE -> HTTP /boot.ipxe -> kernel+initrd.
- HTTP server (axum) with byte-Range ISO streaming and an in-place
  ISO9660 lookup so kernel/initrd are served from inside the ISO
  without ever extracting it to disk.
- Linux ISOs boot via kernel+initrd extraction (memdisk/sanboot fail
  for >1-2 GiB modern distros). Distro-family detection drives the
  cmdline (Debian/Ubuntu, RHEL/Fedora, openSUSE, Arch, Alpine).

## Phase 2 — UX + Windows

- Hierarchical PXE menu (Default / Installers / Tools / Gated
  Deployment) generated from settings — no hand-written .ipxe paths
  surface in the UI. Number-key + letter hotkeys, BIOS+UEFI variants
  for some RHEL ISOs.
- Gated Deployment "horse-race" queue: clients join, operator picks
  one ISO, every gate launches simultaneously via tokio::sync::Notify.
- Bootimus-pattern Windows: WimPatcher injects a CRLF startnet.cmd
  into boot.wim so vanilla WinPE net-uses an SMB share and runs
  setup.exe. All Microsoft-signed; no test certs, no testsigning,
  no httpdisk.sys. SmbManager supervises smbd start/stop/SIGHUP.
- Netbox-style dark UI, fully offline (no CDN, no external fonts).

## Phase 3 — MVP hardening

- TFTP retransmit rewrite with explicit window tracking — UEFI SNP
  clients no longer hang on files that end mid-window. 4 new tests.
- DHCP broadcast-flag honored per RFC 2131 §4.1.
- Multi-arch container (linux/amd64 + linux/arm64). Entrypoint chowns
  bind-mounts as root then drops to uid 10001 via gosu.
- /healthz + /readyz split from /api/status — readyz fails if no
  iPXE binaries are bundled.
- pxeforge seed --from <path> CLI: same pipeline as web upload (slug,
  sha256, introspection, boot-entry).
- All timestamps RFC 3339 (browser Date couldn't parse the 9-tuple).
- Gate poll retains assignment until operator releases — clients that
  retry on transient network errors reuse the assignment instead of
  falling back to the menu.
- Custom OpenShift SCC: hostNetwork + NET_BIND_SERVICE only, no
  NET_RAW.

## Phase 4 — UI restructure + remote storage

- Web UI rebuilt around six tabs inspired by the iVentoy layout:
  Dashboard / Network / Forge Gate / Storage / Terminal / About.
  Old "Monitoring/Content/Configuration" sidebar groups are gone.
- NFS share manager (crates/iso-store/src/nfs.rs): mount NFSv3 or
  NFSv4.1 shares as ISO sources instead of uploading every file
  into the PVC. New IsoSource enum on IsoMeta lets the store resolve
  Local vs NFS lazily. Persisted to <work_dir>/nfs.json; failed
  mounts surface in the UI rather than blocking startup.
- Dockerfile gains nfs-common + iproute2; mounting NFS in-container
  also requires CAP_SYS_ADMIN. Documented in docs/architecture.md.
- LogBus + tracing layer in core: 500-line ring buffer + broadcast
  channel feed an SSE endpoint at /api/log/stream.
- Operator terminal at /api/terminal: whitelisted commands (status,
  isos, clients, gate, nfs, smb, log) — deliberately not a shell.
  Output mirrored onto the LogBus so the live tail and the terminal
  pane share one timeline.
- Network tab: read-only nic_name / subnet_mask / gateway probed
  from `ip` at startup; only DNS server is editable. Editing IP/mask
  on a hot UI would silently break PXE for every client mid-boot.
- Bootimus parity (releases v0.1.55 -> v0.1.62): amber row tint on
  un-bootable ISOs with inline reasons, dashboard "won't boot" panel.

## Tests

56 tests passing across the workspace:
- 16 core (LogBus, gate, settings, arch, client)
- 1 dhcp-proxy (raw option-93 extraction)
- 8 http-api unit (range parsing, terminal split/format)
- 13 http-api integration (gated deployment, range, settings, NFS,
  terminal, log SSE, network endpoint, ui assets, no-external-urls)
- 12 iso-store (introspect, slugify, smb, windows wim, NFS options)
- 6 tftp (RRQ parsing, plan_window edges)

cargo build --workspace and cargo clippy --workspace --all-targets
both finish clean (warnings only, no errors).
This commit is contained in:
Miles Ward
2026-04-29 02:47:00 -04:00
commit cc309da062
67 changed files with 9032 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+290
View File
@@ -0,0 +1,290 @@
# PXEForge
Container-native PXE boot server. A Rust reimplementation of
[iVentoy (ventoy/PXE)](https://github.com/ventoy/PXE), designed from scratch
for Docker/OCI and OpenShift. Upload `.iso` files via the web UI; network
clients PXE-boot them.
> **Status:** Phase 3 MVP. Container image builds and runs, gate flow
> validated end-to-end (two clients join queue → operator assigns in UI →
> both wake within 1 s with the correct boot script). Ready for real
> hardware validation.
## Design non-negotiables
1. **Fully offline / air-gap deployable.** Zero CDN assets. Zero external
HTTP calls from the server, the browser, or the generated iPXE scripts.
Build the container once, run forever disconnected.
2. **iPXE is a backend implementation detail.** No `.ipxe` upload path, no
manual script editing, no iPXE terminology in the UI. Every knob in the
web UI maps to a specific script-generation behavior inside the binary.
3. **The client trust store is off-limits.** No test-signed drivers, no
`bcdedit /set testsigning on`, no certificates injected into WinPE or
the target OS.
## What it does
1. **DHCP proxy** (RFC 4578). Coexists with your existing DHCP server —
never assigns IPs. Listens on UDP 67 + UDP 4011.
2. **TFTP server** (RFC 1350 + RFC 2347/2348/2349/7440 option negotiation)
that serves architecture-specific iPXE binaries to firmware PXE ROMs.
3. **HTTP server** that serves the web UI, the generated iPXE boot scripts,
raw ISOs (with Range), and files inside ISOs without prior extraction.
4. **ISO introspection**: auto-detects the distro family and generates the
appropriate kernel+initrd or wimboot chain. No manual config.
5. **Hierarchical PXE menu** mirroring the Phase 2 spec:
```
Default > Boot from Local HDD
Installers > Linux Installers / Windows Installers
Tools > Utilities / PXEForge Shell / Network Card Info
Gated Deployment
```
6. **Gated Deployment queue** — the "horse race gate" flow. A client that
selects *Gated Deployment* gets a numbered position and waits. The
operator picks an ISO in the web UI and fires it to every waiting
client simultaneously.
7. **Web UI** (Netbox-style): sidebar nav (Dashboard / Clients / Gated
Deployment / Images / Settings / About), top tabs, dark theme, teal
accents. All assets served from the binary — no external requests.
8. **Settings API** lets you change the default boot-menu timeout (default
600s), the timeout action (stay / Local HDD / Gated Deployment), and
feature toggles like Windows ISO support. The iPXE scripts regenerate
on every request using current settings.
### Architectures supported on day one
| DHCP option 93 | Architecture | Binary served |
|----------------|-----------------|-------------------------|
| `0x0000` | Legacy x86 BIOS | `undionly.kpxe` |
| `0x0006` | IA32 UEFI | `snponly-i386.efi` |
| `0x0007`/`0x0009` | x86_64 UEFI | `snponly.efi` |
| `0x000B` | ARM64 UEFI | `snponly-arm64.efi` |
UEFI firmware that sends `HTTPClient` in option 60 is handled too — we
skip TFTP and respond with an HTTP URL.
## Quick start — MVP container (recommended)
```bash
# 1. Pull bundled iPXE binaries (~2 MB, one-time).
./scripts/fetch-ipxe.sh
# 2. Build the container image (~3 min first time).
docker buildx build -f deploy/docker/Dockerfile -t pxeforge:0.1.0 --load .
# 3. Run it on the box plugged into your PXE network. Set PUBLIC_IP to
# this host's LAN address so advertised iPXE URLs are reachable.
docker run -d --name pxeforge \
--network host \
-e PXEFORGE_PUBLIC_IP=10.0.0.5 \
-e PXEFORGE_DHCP_MODE=proxy \
-v $PWD/data/isos:/var/lib/pxeforge/isos \
-v $PWD/data/work:/var/lib/pxeforge/work \
pxeforge:0.1.0
# 4. Open the UI and drop an ISO in.
open http://10.0.0.5
```
Host networking is required in proxy mode so the container sees DHCPDISCOVER
broadcasts from the PXE VLAN. On macOS/Windows hosts Docker runs in a Linux
VM, so "host" means the VM — use `pxeforge-dev` in `docker-compose.yml` for
API-only testing on a laptop.
### Quick start — docker compose
```bash
# MVP / API testing on a laptop (no DHCP, high ports):
PXEFORGE_PUBLIC_IP=127.0.0.1 docker compose up pxeforge-dev
# Real PXE deployment on a Linux host (host network, DHCP proxy on):
PXEFORGE_PUBLIC_IP=10.0.0.5 docker compose up pxeforge
```
### Multi-arch build + push
For deploying to x86_64 servers, build both arches in one manifest:
```bash
# One-time: bootstrap a multi-arch builder.
docker buildx create --name pxeforge-multi --driver docker-container --use
# Build + push both linux/amd64 and linux/arm64 under one tag.
docker buildx build --builder pxeforge-multi \
--platform linux/amd64,linux/arm64 \
-t ghcr.io/YOUR-ORG/pxeforge:0.1.0 \
--push \
-f deploy/docker/Dockerfile .
```
On an Apple Silicon host, the amd64 stage runs under QEMU emulation (~10-15 min for a cold cache). On a Linux x86_64 host, both arches build natively at normal speed. CI runners on GitHub Actions with `docker/build-push-action@v5` handle this cleanly.
### Build from source (no container)
```bash
./scripts/fetch-ipxe.sh
cargo run --release # needs NET_BIND_SERVICE or root for :80/:69
```
### Container health probes
| Endpoint | Purpose |
|-------------|---------------------------------------------------------------|
| `/healthz` | Liveness — HTTP stack alive. Always 200. |
| `/readyz` | Readiness — 200 only if iPXE binaries bundled + ISO dir OK. |
| `/api/status` | Full JSON status: versions, assets, counts, live settings, SMB state. |
### Pre-seeding ISOs from a directory
For CI, pre-baked homelab deployments, or a fresh PVC, the binary has a
`seed` subcommand that imports every `*.iso` from a host path through the
same pipeline the web UI uses (introspection + boot-entry generation):
```bash
docker run --rm \
-v /my/iso-library:/seed:ro \
-v pxeforge-data:/var/lib/pxeforge/isos \
-e PXEFORGE_PUBLIC_IP=10.0.0.5 \
pxeforge:0.1.0 seed --from /seed
# Dry run first to see what would be imported:
docker run --rm -v /my/iso-library:/seed:ro pxeforge:0.1.0 seed --from /seed --dry-run
```
### Environment overrides
| Var | Default | Meaning |
|------------------------|-----------------------------|----------------------------------------|
| `PXEFORGE_HTTP_PORT` | `80` | Web UI + boot script HTTP port |
| `PXEFORGE_TFTP_PORT` | `69` | TFTP port |
| `PXEFORGE_DHCP_PORT` | `67` | DHCP server-side port |
| `PXEFORGE_DHCP_MODE` | `proxy` | `proxy` or `disabled` |
| `PXEFORGE_PUBLIC_IP` | auto-detect | Advertised IP for clients. Startup **fails** if unset and auto-detect returns loopback. |
| `PXEFORGE_ISO_DIR` | `/var/lib/pxeforge/isos` | Where uploaded ISOs live |
| `PXEFORGE_WORK_DIR` | `/var/lib/pxeforge/work` | Scratch + runtime settings |
| `PXEFORGE_LOG` | `info,pxeforge=debug` | `tracing` filter |
## What the boot menu looks like on a real client
```
PXEForge - network boot menu
------------------------- Default -------------------------
Boot from Local HDD
----------------------- Installers -----------------------
Linux Installers >
Windows Installers > (only if enabled in Settings)
-------------------------- Tools --------------------------
Tools > Utilities / Shell /
NIC Info / Reboot /
Exit and continue BIOS
---------------------- Gated Deployment ------------------
Gated Deployment (join queue)
```
Linux/Windows submenus show file sizes iVentoy-style:
```
PXEForge - Linux Installers
[ 4376 MB] CentOS-7-x86_64-DVD-1810
[ 2002 MB] Fedora-Workstation-Live-x86_64-38-1.6
[ 4699 MB] ubuntu-22.04.2-desktop-amd64
< Back to main menu
```
iPXE never appears in the UI — the whole hierarchy above is generated from
ISOs you upload via drag-and-drop in the web UI plus toggles in Settings.
## OpenShift
```bash
oc apply -f deploy/openshift/
oc -n pxeforge get all
oc -n pxeforge get route pxeforge -o jsonpath='{.spec.host}'
```
### Why a custom SCC?
The default `restricted-v2` blocks `hostNetwork` and all capabilities. PXE
cannot work without host network (CNI overlays don't deliver L2 broadcast
into pod netns), and we need `NET_BIND_SERVICE` to bind <1024. The custom
`pxeforge-scc` grants exactly those two and nothing else. No raw sockets,
no privileged mode — proxy-mode DHCP sidesteps the usual requirements.
### What's on host ports
| Port | Proto | Purpose |
|----------|-------|---------------------------------|
| 67 | UDP | DHCP server (proxy replies) |
| 69 | UDP | TFTP |
| 4011 | UDP | PXE Boot Server discovery |
| 80 | TCP | Web UI + HTTP boot assets |
The OpenShift Route only covers 80/TCP. Clients on the PXE network talk to
the node's host IP directly for UDP.
## Windows support
Enabled by toggling **Windows ISO support** under Settings. The flow:
1. Upload a stock Microsoft Windows install ISO (vanilla, no pre-processing).
2. On upload, PXEForge extracts the ISO and uses `wimlib-imagex` to rewrite
image index 2 (WinPE) of `sources/boot.wim`. It injects exactly two
plain-text files:
- `Windows/System32/winpeshl.ini` — tells WinPE to run `startnet.cmd`.
- `Windows/System32/startnet.cmd` — runs `wpeinit`, waits for the SMB
host to be reachable, `net use Z: \\<server>\<share> /user:guest`,
then `Z:\setup.exe`.
3. The container's Samba `smbd` serves the extracted install tree on :445.
4. The client gets chainloaded into wimboot → patched WinPE → Windows Setup
running off the SMB share. **Every binary the client executes is stock
Microsoft-signed.**
### What we never do
- Ship drivers — signed, test-signed, or otherwise — that load on the client.
- Install certificates into the target's trust store or WinPE boot policy.
- Recommend `bcdedit /set testsigning on` or any equivalent signing-policy
weakening.
### Credit & limitations
The SMB-based approach is adapted from [Bootimus](https://github.com/garybowers/bootimus)
(Apache-2.0). Re-implemented in Rust; no code was copied verbatim. Known
operational constraints inherited from the design:
- **Port 445 must be directly reachable from PXE clients.** `net use`
ignores alternate ports. In OpenShift this means `hostPort: 445` on the
deployment; on a host that already runs SMB it will collide.
- Windows 10/11 client SKUs are the tested target. Server SKUs untested.
- Hardware with NICs/storage controllers missing from WinPE's bundled
drivers will need a driver-pack injection step (not yet implemented).
## Gated Deployment
The "horse race gate" flow, end to end:
1. A client boots and picks **Gated Deployment** in the PXE menu (or falls
through on timeout with the default `timeout_action`).
2. The client joins the queue, gets a numbered gate position, and enters a
long-poll loop (25s per request, auto-renewed).
3. In the web UI's **Gated Deployment** tab, the operator sees each waiting
client with its MAC, IP, arch, and position.
4. The operator selects an image and clicks **Launch for all waiting**.
The server broadcasts the assignment to every gated client via a
`tokio::sync::Notify`; each client's next poll returns the boot script
for the chosen image.
5. Every client chains the same image at effectively the same moment — the
gate opens and the horses run together.
No user-facing iPXE anywhere in this flow. The client only ever runs
scripts we generate; the operator only interacts with the web UI.
## Architecture
See [`docs/architecture.md`](docs/architecture.md) for the protocol stack,
crate layout, and the full decision log.
## Licence
MIT OR Apache-2.0.