303 lines
13 KiB
Markdown
303 lines
13 KiB
Markdown
# OpenPXE
|
||
|
||
Container-native PXE boot server. A Rust reimplementation of
|
||
[iVentoy (ventoy/PXE)](https://github.com/ventoy/PXE), designed from scratch
|
||
for Docker/OCI and OpenShift. Upload `.iso` files via the web UI; network
|
||
clients PXE-boot them.
|
||
|
||
> **Status:** v0.3.2 / pre-beta. Phases 1–5 complete: full PXE stack,
|
||
> Queued Deployment queue, NFS-share ISO sources, live tracing log + an
|
||
> operator terminal, per-MAC host bindings (Tinkerbell-style),
|
||
> Prometheus `/metrics`, light/dark theme toggle, animated OpenPXE
|
||
> imaging-progress widget, and per-ISO boot passwords. The test suite and
|
||
> clippy are part of the release checklist. Ready for real-hardware validation.
|
||
|
||
## Design non-negotiables
|
||
|
||
1. **Fully offline / air-gap deployable.** Zero CDN assets. Zero external
|
||
HTTP calls from the server, the browser, or the generated iPXE scripts.
|
||
Build the container once, run forever disconnected.
|
||
2. **iPXE is a backend implementation detail.** No `.ipxe` upload path, no
|
||
manual script editing, no iPXE terminology in the UI. Every knob in the
|
||
web UI maps to a specific script-generation behavior inside the binary.
|
||
3. **The client trust store is off-limits.** No test-signed drivers, no
|
||
`bcdedit /set testsigning on`, no certificates injected into WinPE or
|
||
the target OS.
|
||
|
||
## What it does
|
||
|
||
1. **DHCP proxy** (RFC 4578). Coexists with your existing DHCP server —
|
||
never assigns IPs. Listens on UDP 67 + UDP 4011.
|
||
2. **TFTP server** (RFC 1350 + RFC 2347/2348/2349/7440 option negotiation)
|
||
that serves architecture-specific iPXE binaries to firmware PXE ROMs.
|
||
3. **HTTP server** that serves the web UI, the generated iPXE boot scripts,
|
||
raw ISOs (with Range), and files inside ISOs without prior extraction.
|
||
4. **ISO introspection**: auto-detects the distro family and generates the
|
||
appropriate kernel+initrd or wimboot chain. No manual config.
|
||
5. **Hierarchical PXE menu** mirroring the Phase 2 spec:
|
||
```
|
||
Default > Boot from Local HDD
|
||
Installers > Linux Installers / Windows Installers
|
||
Tools > Utilities / OpenPXE Shell / Network Card Info
|
||
Queued Deployment
|
||
```
|
||
6. **Queued Deployment queue** — the coordinated launch flow. A client that
|
||
selects *Queued Deployment* gets a numbered position and waits. The
|
||
operator picks an ISO in the web UI and fires it to every waiting
|
||
client simultaneously.
|
||
7. **Web UI** (Netbox-style): sidebar nav (Dashboard / Network / Queue /
|
||
Storage / Hosts / Terminal / About), light + dark themes
|
||
(toggle top-right or press `T`), animated OpenPXE progress
|
||
widget when devices are imaging. All assets served from the binary —
|
||
no external requests.
|
||
8. **Per-MAC host bindings.** Pin a MAC to a boot target and the client
|
||
skips the menu, chains straight through. Inspired by Tinkerbell's
|
||
`smee` MAC-prepended URL pattern.
|
||
9. **Prometheus metrics** at `/metrics` — DHCP replies by arch, TFTP
|
||
transfer counts and bytes, HTTP request counts by route, queue /
|
||
imaging gauges, uptime, build info. Plain text exposition format,
|
||
no external metrics framework dependency.
|
||
8. **Settings API** lets you change the default boot-menu timeout (default
|
||
600s), the timeout action (stay / Local HDD / Queued Deployment), and
|
||
feature toggles like Windows ISO support. The iPXE scripts regenerate
|
||
on every request using current settings.
|
||
|
||
### Architectures supported on day one
|
||
|
||
| DHCP option 93 | Architecture | Binary served |
|
||
|----------------|-----------------|-------------------------|
|
||
| `0x0000` | Legacy x86 BIOS | `undionly.kpxe` |
|
||
| `0x0006` | IA32 UEFI | `snponly-i386.efi` |
|
||
| `0x0007`/`0x0009` | x86_64 UEFI | `snponly.efi` |
|
||
| `0x000B` | ARM64 UEFI | `snponly-arm64.efi` |
|
||
|
||
UEFI firmware that sends `HTTPClient` in option 60 is handled too — we
|
||
skip TFTP and respond with an HTTP URL.
|
||
|
||
## Quick start — MVP container (recommended)
|
||
|
||
```bash
|
||
# 1. Pull bundled iPXE binaries (~2 MB, one-time).
|
||
./scripts/fetch-ipxe.sh
|
||
|
||
# 2. Build the container image (~3 min first time).
|
||
docker buildx build -f deploy/docker/Dockerfile -t openpxe:0.3.2 --load .
|
||
|
||
# 3. Run it on the box plugged into your PXE network. Set PUBLIC_IP to
|
||
# this host's LAN address so advertised iPXE URLs are reachable.
|
||
docker run -d --name openpxe \
|
||
--network host \
|
||
-e OPENPXE_PUBLIC_IP=10.0.0.5 \
|
||
-e OPENPXE_DHCP_MODE=proxy \
|
||
-v $PWD/data/isos:/var/lib/openpxe/isos \
|
||
-v $PWD/data/work:/var/lib/openpxe/work \
|
||
openpxe:0.3.2
|
||
|
||
# 4. Open the UI and drop an ISO in.
|
||
open http://10.0.0.5
|
||
```
|
||
|
||
Host networking is required in proxy mode so the container sees DHCPDISCOVER
|
||
broadcasts from the PXE VLAN. On macOS/Windows hosts Docker runs in a Linux
|
||
VM, so "host" means the VM — use `openpxe-dev` in `docker-compose.yml` for
|
||
API-only testing on a laptop.
|
||
|
||
### Quick start — docker compose
|
||
|
||
```bash
|
||
# MVP / API testing on a laptop (no DHCP, high ports):
|
||
OPENPXE_PUBLIC_IP=127.0.0.1 docker compose up openpxe-dev
|
||
# Real PXE deployment on a Linux host (host network, DHCP proxy on):
|
||
OPENPXE_PUBLIC_IP=10.0.0.5 docker compose up openpxe
|
||
```
|
||
|
||
### Multi-arch build + push
|
||
|
||
For deploying to x86_64 servers, build both arches in one manifest:
|
||
|
||
```bash
|
||
# One-time: bootstrap a multi-arch builder.
|
||
docker buildx create --name openpxe-multi --driver docker-container --use
|
||
|
||
# Build + push both linux/amd64 and linux/arm64 under one tag.
|
||
docker buildx build --builder openpxe-multi \
|
||
--platform linux/amd64,linux/arm64 \
|
||
-t ghcr.io/YOUR-ORG/openpxe:0.3.2 \
|
||
--push \
|
||
-f deploy/docker/Dockerfile .
|
||
```
|
||
|
||
On an Apple Silicon host, the amd64 stage runs under QEMU emulation (~10-15 min for a cold cache). On a Linux x86_64 host, both arches build natively at normal speed. CI runners on GitHub Actions with `docker/build-push-action@v5` handle this cleanly.
|
||
|
||
### Build from source (no container)
|
||
|
||
```bash
|
||
./scripts/fetch-ipxe.sh
|
||
cargo run --release # needs NET_BIND_SERVICE or root for :80/:69
|
||
```
|
||
|
||
### Container health probes
|
||
|
||
| Endpoint | Purpose |
|
||
|-------------|---------------------------------------------------------------|
|
||
| `/healthz` | Liveness — HTTP stack alive. Always 200. |
|
||
| `/readyz` | Readiness — 200 only if iPXE binaries bundled + ISO dir OK. |
|
||
| `/api/status` | Full JSON status: versions, assets, counts, live settings, SMB state. |
|
||
|
||
### Pre-seeding ISOs from a directory
|
||
|
||
For CI, pre-baked homelab deployments, or a fresh PVC, the binary has a
|
||
`seed` subcommand that imports every `*.iso` from a host path through the
|
||
same pipeline the web UI uses (introspection + boot-entry generation):
|
||
|
||
```bash
|
||
docker run --rm \
|
||
-v /my/iso-library:/seed:ro \
|
||
-v openpxe-data:/var/lib/openpxe/isos \
|
||
-e OPENPXE_PUBLIC_IP=10.0.0.5 \
|
||
openpxe:0.3.2 seed --from /seed
|
||
|
||
# Dry run first to see what would be imported:
|
||
docker run --rm -v /my/iso-library:/seed:ro openpxe:0.3.2 seed --from /seed --dry-run
|
||
```
|
||
|
||
### Environment overrides
|
||
|
||
| Var | Default | Meaning |
|
||
|------------------------|-----------------------------|----------------------------------------|
|
||
| `OPENPXE_HTTP_PORT` | `80` | Web UI + boot script HTTP port |
|
||
| `OPENPXE_TFTP_PORT` | `69` | TFTP port |
|
||
| `OPENPXE_DHCP_PORT` | `67` | DHCP server-side port |
|
||
| `OPENPXE_DHCP_MODE` | `proxy` | `proxy` or `disabled` |
|
||
| `OPENPXE_PUBLIC_IP` | auto-detect | Advertised IP for clients. Startup **fails** if unset and auto-detect returns loopback. |
|
||
| `OPENPXE_ISO_DIR` | `/var/lib/openpxe/isos` | Where uploaded ISOs live |
|
||
| `OPENPXE_WORK_DIR` | `/var/lib/openpxe/work` | Scratch + runtime settings |
|
||
| `OPENPXE_LOG` | `info,openpxe=debug` | `tracing` filter |
|
||
|
||
## What the boot menu looks like on a real client
|
||
|
||
```
|
||
OpenPXE - network boot menu
|
||
|
||
------------------------- Default -------------------------
|
||
Boot from Local HDD
|
||
----------------------- Installers -----------------------
|
||
Linux Installers >
|
||
Windows Installers > (only if enabled in Settings)
|
||
-------------------------- Tools --------------------------
|
||
Tools > Utilities / Shell /
|
||
NIC Info / Reboot /
|
||
Exit and continue BIOS
|
||
---------------------- Queued Deployment ------------------
|
||
Queued Deployment (join queue)
|
||
```
|
||
|
||
Linux/Windows submenus show file sizes iVentoy-style:
|
||
|
||
```
|
||
OpenPXE - Linux Installers
|
||
|
||
[ 4376 MB] CentOS-7-x86_64-DVD-1810
|
||
[ 2002 MB] Fedora-Workstation-Live-x86_64-38-1.6
|
||
[ 4699 MB] ubuntu-22.04.2-desktop-amd64
|
||
< Back to main menu
|
||
```
|
||
|
||
iPXE never appears in the UI — the whole hierarchy above is generated from
|
||
ISOs you upload via drag-and-drop in the web UI plus toggles in Settings.
|
||
|
||
## OpenShift
|
||
|
||
```bash
|
||
oc apply -f deploy/openshift/
|
||
oc -n openpxe get all
|
||
oc -n openpxe get route openpxe -o jsonpath='{.spec.host}'
|
||
```
|
||
|
||
### Why a custom SCC?
|
||
|
||
The default `restricted-v2` blocks `hostNetwork` and all capabilities. PXE
|
||
cannot work without host network (CNI overlays don't deliver L2 broadcast
|
||
into pod netns), and we need `NET_BIND_SERVICE` to bind <1024. The custom
|
||
`openpxe-scc` grants exactly those two and nothing else. No raw sockets,
|
||
no privileged mode — proxy-mode DHCP sidesteps the usual requirements.
|
||
|
||
### What's on host ports
|
||
|
||
| Port | Proto | Purpose |
|
||
|----------|-------|---------------------------------|
|
||
| 67 | UDP | DHCP server (proxy replies) |
|
||
| 69 | UDP | TFTP |
|
||
| 4011 | UDP | PXE Boot Server discovery |
|
||
| 80 | TCP | Web UI + HTTP boot assets |
|
||
|
||
The OpenShift Route only covers 80/TCP. Clients on the PXE network talk to
|
||
the node's host IP directly for UDP.
|
||
|
||
## Windows support
|
||
|
||
Enabled by toggling **Windows ISO support** under Settings. The flow:
|
||
|
||
1. Upload a stock Microsoft Windows install ISO (vanilla, no pre-processing).
|
||
2. On upload, OpenPXE extracts the ISO and uses `wimlib-imagex` to rewrite
|
||
image index 2 (WinPE) of `sources/boot.wim`. It injects exactly two
|
||
plain-text files:
|
||
- `Windows/System32/winpeshl.ini` — tells WinPE to run `startnet.cmd`.
|
||
- `Windows/System32/startnet.cmd` — runs `wpeinit`, waits for the SMB
|
||
host to be reachable, `net use Z: \\<server>\<share> /user:guest`,
|
||
then `Z:\setup.exe`.
|
||
3. The container's Samba `smbd` serves the extracted install tree on :445.
|
||
4. The client gets chainloaded into wimboot → patched WinPE → Windows Setup
|
||
running off the SMB share. **Every binary the client executes is stock
|
||
Microsoft-signed.**
|
||
|
||
### What we never do
|
||
|
||
- Ship drivers — signed, test-signed, or otherwise — that load on the client.
|
||
- Install certificates into the target's trust store or WinPE boot policy.
|
||
- Recommend `bcdedit /set testsigning on` or any equivalent signing-policy
|
||
weakening.
|
||
|
||
### Credit & limitations
|
||
|
||
The SMB-based approach is adapted from [Bootimus](https://github.com/garybowers/bootimus)
|
||
(Apache-2.0). Re-implemented in Rust; no code was copied verbatim. Known
|
||
operational constraints inherited from the design:
|
||
|
||
- **Port 445 must be directly reachable from PXE clients.** `net use`
|
||
ignores alternate ports. In OpenShift this means `hostPort: 445` on the
|
||
deployment; on a host that already runs SMB it will collide.
|
||
- Windows 10/11 client SKUs are the tested target. Server SKUs untested.
|
||
- Hardware with NICs/storage controllers missing from WinPE's bundled
|
||
drivers will need a driver-pack injection step (not yet implemented).
|
||
|
||
## Queued Deployment
|
||
|
||
The coordinated launch flow, end to end:
|
||
|
||
1. A client boots and picks **Queued Deployment** in the PXE menu (or falls
|
||
through on timeout with the default `timeout_action`).
|
||
2. The client joins the queue, gets a numbered queue position, and enters a
|
||
long-poll loop (25s per request, auto-renewed).
|
||
3. In the web UI's **Queued Deployment** tab, the operator sees each waiting
|
||
client with its MAC, IP, arch, and position.
|
||
4. The operator selects an image and clicks **Launch for all waiting**.
|
||
The server broadcasts the assignment to every queued client via a
|
||
`tokio::sync::Notify`; each client's next poll returns the boot script
|
||
for the chosen image.
|
||
5. Every client chains the same image at effectively the same moment. The
|
||
queue stays visible until the operator releases entries, which keeps a
|
||
useful audit trail during hardware testing.
|
||
|
||
No user-facing iPXE anywhere in this flow. The client only ever runs
|
||
scripts we generate; the operator only interacts with the web UI.
|
||
|
||
## Architecture
|
||
|
||
See [`docs/architecture.md`](docs/architecture.md) for the protocol stack,
|
||
crate layout, and the full decision log.
|
||
|
||
## Licence
|
||
|
||
MIT OR Apache-2.0.
|