Name update
This commit is contained in:
Miles Ward
@@ -0,0 +1,270 @@
|
||||
# PXEForge architecture
|
||||
|
||||
## Protocol stack
|
||||
|
||||
```
|
||||
Client firmware PXE ROM
|
||||
│
|
||||
│ DHCPDISCOVER (UDP/67 broadcast, option 60 "PXEClient", option 93 arch)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PXEForge │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
|
||||
│ │ DHCP proxy │ │ TFTP server │ │ HTTP server (axum) │ │
|
||||
│ │ :67, :4011 │ │ :69 │ │ :80 │ │
|
||||
│ │ (dhcproto) │ │ (custom) │ │ │ │
|
||||
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────────────┘ │
|
||||
│ │ │ │ │
|
||||
│ └────────────────────┼─────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────────▼────────────┐ │
|
||||
│ │ IsoStore (on disk) │ │
|
||||
│ │ + ClientRegistry │ │
|
||||
│ └───────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼ reply with option 60, option 66 tftp-server, option 67 bootfile
|
||||
Client firmware PXE ROM
|
||||
│
|
||||
│ TFTP RRQ: snponly.efi (or undionly.kpxe for BIOS)
|
||||
▼
|
||||
Client runs iPXE
|
||||
│
|
||||
│ DHCPDISCOVER with option 77 "iPXE"
|
||||
▼
|
||||
PXEForge sees user-class "iPXE" → replies with HTTP URL: /boot.ipxe
|
||||
│
|
||||
│ HTTP GET /boot.ipxe (iPXE menu, auto-generated from IsoStore)
|
||||
▼
|
||||
User picks entry; iPXE chains /boot/<id>.ipxe
|
||||
│
|
||||
│ HTTP GET kernel + initrd (or wimboot + WIM files)
|
||||
▼
|
||||
Kernel boots with distro-specific args pointing back at /iso/<id>.iso
|
||||
```
|
||||
|
||||
## Crate layout
|
||||
|
||||
| Crate | Responsibility |
|
||||
|---------------------|-------------------------------------------------------------------|
|
||||
| `pxeforge-core` | Shared types: `Config`, `ClientArch`, `FirmwareClass`, `ClientRegistry` |
|
||||
| `pxeforge-ipxe-assets` | Embeds bundled iPXE binaries via `rust-embed` |
|
||||
| `pxeforge-iso-store` | On-disk ISO store, introspection, boot-entry generation |
|
||||
| `pxeforge-dhcp-proxy` | UDP listener + `dhcproto` reply builder; pure `decide()` unit-testable |
|
||||
| `pxeforge-tftp` | RFC 1350 + OACK (blksize / tsize / windowsize). Serves only embedded assets — no filesystem |
|
||||
| `pxeforge-http-api` | `axum` router: web UI, API, iPXE script generation, ISO streaming |
|
||||
| `pxeforge-webui` | Single `index.html` served as static string |
|
||||
| `pxeforge` (bin) | Wires everything together, runs the three servers concurrently |
|
||||
|
||||
## Key decisions and why
|
||||
|
||||
### DHCP proxy only, not a full DHCP server
|
||||
|
||||
Proxy mode (RFC 4578) replies with boot parameters (`siaddr`, option 66, 67)
|
||||
but never sets `yiaddr` — it does not lease IPs. The client merges proxy
|
||||
replies with its normal DHCP lease from the network's existing DHCP server.
|
||||
|
||||
This sidesteps needing `CAP_NET_RAW` or AF_PACKET. A full DHCP server has
|
||||
to craft Ethernet frames to a client that doesn't yet have an IP — that
|
||||
requires raw sockets and elevated privileges. A proxy replies on UDP to a
|
||||
client that already has an IP (or will shortly, from the other DHCP server),
|
||||
so plain `SOCK_DGRAM` is enough.
|
||||
|
||||
### Two-stage iPXE chain
|
||||
|
||||
1. Firmware PXE ROM sends DHCPDISCOVER with option 60 = `PXEClient`,
|
||||
option 93 = arch.
|
||||
2. PXEForge replies with TFTP server + arch-specific iPXE binary
|
||||
(`undionly.kpxe` for Legacy BIOS, `snponly.efi` for x86_64 UEFI, etc.).
|
||||
3. Client TFTPs the iPXE binary and runs it.
|
||||
4. iPXE does its own DHCP, setting option 77 (user-class) to `iPXE`.
|
||||
5. PXEForge detects the user-class and this time replies with an HTTP URL
|
||||
in option 67 pointing at `/boot.ipxe`.
|
||||
6. iPXE fetches and executes that script, which chains the selected OS.
|
||||
|
||||
The split is essential because firmware PXE ROMs only speak TFTP; they
|
||||
don't do HTTP. iPXE adds HTTP (and a lot more), and is small enough to fit
|
||||
in the TFTP hop.
|
||||
|
||||
### TFTP option negotiation is mandatory
|
||||
|
||||
Default 512-byte blocks means an ~1 MiB iPXE binary is ~2000 packets.
|
||||
On anything less than pristine wired Ethernet this is unusably slow or
|
||||
outright fails. We negotiate:
|
||||
|
||||
- `blksize` (RFC 2348): up to ~1468 bytes for Ethernet MTU
|
||||
- `tsize` (RFC 2349): file size; some PXE ROMs require it present
|
||||
- `windowsize` (RFC 7440): 8–16 gives order-of-magnitude throughput gains
|
||||
|
||||
### Do not touch the client OS trust store
|
||||
|
||||
Whatever we do for Windows, we never:
|
||||
|
||||
- ship drivers signed with test/development certificates
|
||||
- instruct users to enable `bcdedit /set testsigning on`
|
||||
- install any certificate into the target's root/trust store
|
||||
|
||||
iVentoy's `httpdisk.sys` approach broke this rule. PXEForge doesn't.
|
||||
|
||||
### Linux ISO boot uses kernel+initrd extraction, not sanboot
|
||||
|
||||
Loading a full ISO into RAM via `memdisk` or `sanboot` fails:
|
||||
- Above ~1–2 GiB, RAM emulation is too slow or too large.
|
||||
- Most modern distros can't find the emulated CD device from the initramfs.
|
||||
|
||||
We instead extract `vmlinuz` + `initrd` at upload time (well, look up their
|
||||
locations — we serve them via ISO9660 byte-range lookup, no extraction on
|
||||
disk) and pass distro-specific kernel args that point the installer back at
|
||||
the HTTP-served ISO.
|
||||
|
||||
### No iPXE UX
|
||||
|
||||
iPXE is an implementation detail. The web UI only accepts `.iso` uploads
|
||||
and shows distros, not `.ipxe` scripts or boot targets. If someone who
|
||||
knows iPXE wants to peek, they can `curl /boot.ipxe` — that's fine. But
|
||||
the UI never surfaces it.
|
||||
|
||||
## Unit tests
|
||||
|
||||
Run with `cargo test --workspace --lib`. Current coverage:
|
||||
|
||||
- `ClientArch` alias handling (0x0009 → x86_64 UEFI)
|
||||
- `FirmwareClass` classification (iPXE wins over PXEClient echo)
|
||||
- Raw option-93 extraction from the wire
|
||||
- TFTP RRQ parsing with options
|
||||
- HTTP Range header parsing (full, open-ended, suffix, explicit)
|
||||
- Distro family detection from volume label
|
||||
- ISO id slugification
|
||||
|
||||
## Phase 3 hardening
|
||||
|
||||
A code-review pass after Phase 2 turned up 16 issues across the protocol
|
||||
stack and container surface. Everything P0/P1/P2 is addressed in this
|
||||
release:
|
||||
|
||||
**Protocol reliability** (review P1 #10/#12):
|
||||
- TFTP retransmit loop rewritten with explicit window tracking so UEFI SNP
|
||||
clients no longer hang on files that end mid-window. Covered by 4 new
|
||||
unit tests (`plan_window_*` in `crates/tftp`).
|
||||
- DHCP proxy now honors the broadcast-flag bit (RFC 2131 §4.1) when
|
||||
choosing unicast vs broadcast reply destination.
|
||||
|
||||
**Container posture** (review P1 #2/#13/#14/#15):
|
||||
- Multi-arch container (linux/amd64 + linux/arm64) via `buildx`.
|
||||
- New entrypoint (`deploy/docker/entrypoint.sh`) chowns data dirs as root,
|
||||
then drops to uid 10001 via `gosu` — fixes the "bind-mount comes up
|
||||
root-owned" problem that breaks ISO upload on standard Docker hosts.
|
||||
- `/healthz` and `/readyz` split from `/api/status` — readyz fails if no
|
||||
iPXE binaries are bundled, giving K8s probes a real signal.
|
||||
- Startup aborts with a clear error if `PXEFORGE_PUBLIC_IP` can't be
|
||||
auto-detected (no more silent `127.0.0.1` advertisement).
|
||||
- `scripts/fetch-ipxe.sh` fails non-zero if zero binaries download; the
|
||||
Dockerfile uses arch-scoped paths (`x86_64-efi/snponly.efi` etc.).
|
||||
|
||||
**Windows boot plumbing** (new):
|
||||
- `pxeforge-iso-store::smb::SmbManager` supervises `smbd` on the Windows
|
||||
toggle: `start` → spawn + write `smb.conf`; `reconcile` → SIGHUP on
|
||||
share changes; `stop` → SIGTERM. `SmbState` surfaced to the UI for
|
||||
visibility.
|
||||
- `extract_windows_iso` shells out to `7z` (or `bsdtar` fallback) to
|
||||
unpack the ISO tree into `smb_dir/<slug>/` as the SMB share root.
|
||||
- `WimPatcher` (from Phase 2) is still in place for `boot.wim` injection.
|
||||
- Guardrail: flipping the Windows toggle in `/api/settings` is rejected
|
||||
with a 400 if `wimboot` isn't bundled.
|
||||
|
||||
**iVentoy/Bootimus parity** (P2):
|
||||
- ISO sizes in menu labels (`[ 4376 MB]`), iVentoy format.
|
||||
- `Reboot Computer` + `Exit and continue BIOS boot` in Tools menu.
|
||||
- Number-key hotkeys (1..9) on boot entries, letter hotkeys on tools.
|
||||
- Clients tab cross-joins the gate queue so an operator sees "at gate #2"
|
||||
or "assigned: ubuntu-linux" status inline.
|
||||
|
||||
**Developer ergonomics** (new):
|
||||
- `pxeforge seed --from <path>` CLI to import ISOs from a directory.
|
||||
Same pipeline as web upload (slug, sha256, introspection, boot-entry).
|
||||
- `docker-compose.yml` with `pxeforge` (host network, real PXE) and
|
||||
`pxeforge-dev` (published ports, DHCP disabled, for API testing).
|
||||
|
||||
**API cleanliness**:
|
||||
- All timestamps now serialized as RFC 3339 strings (the `time` crate's
|
||||
default 9-tuple broke browser `Date` parsing).
|
||||
- Gate poll retains assignment until the operator releases it; if the
|
||||
client's chain fails, it reuses the assignment instead of falling back
|
||||
to the menu.
|
||||
|
||||
## Phase 4 — UI restructure + remote storage
|
||||
|
||||
The web UI was rebuilt around six tabs (Dashboard / Network / Forge Gate /
|
||||
Storage / Terminal / About) inspired by the iVentoy layout the user
|
||||
attached and Netbox Labs's compact-card pattern. The old hierarchical
|
||||
"Monitoring / Content / Configuration" sidebar grouping is gone — every
|
||||
tab is one click from the brand bar.
|
||||
|
||||
**NFS share manager** (`crates/iso-store/src/nfs.rs`):
|
||||
- Operators add a remote share via Storage → NFS shares; PXEForge mounts
|
||||
it under `<work_dir>/nfs/<id>/` and walks it for `*.iso` files.
|
||||
- Each ISO found is registered with `IsoStore::register_external` using
|
||||
a new `IsoSource::Nfs { mount_id, relative_path }` variant. The store
|
||||
resolves these to disk lazily via `set_nfs_root`, so adding NFS
|
||||
required exactly one new field on `IsoMeta` (with a serde default for
|
||||
forward compatibility with old `meta.json` files).
|
||||
- Versions: `vers=3` and `vers=4.1` only. v3 also gets `nolock` since
|
||||
appliances commonly disable `lockd`. All mounts use `soft,timeo=100`
|
||||
so a dead server surfaces as a UI error rather than wedging iPXE.
|
||||
- State persists to `<work_dir>/nfs.json`. On startup the manager
|
||||
re-attempts every spec; failures are logged per-mount and surfaced in
|
||||
the UI rather than blocking startup.
|
||||
- Container requirement: `mount.nfs` binary plus `CAP_SYS_ADMIN`. The
|
||||
default Dockerfile bundles `nfs-common`. OpenShift operators must
|
||||
swap to a more permissive SCC or use a CSI driver.
|
||||
|
||||
**Live log + operator terminal** (`crates/core/src/log_bus.rs`,
|
||||
`crates/http-api/src/{log_stream,terminal}.rs`):
|
||||
- A `LogBusLayer` feeds every tracing event into a 500-line ring buffer
|
||||
plus a `tokio::sync::broadcast` channel.
|
||||
- `/api/log/stream` is an SSE endpoint that emits the recent buffer
|
||||
followed by live updates. Slow clients see a `lagged` event rather
|
||||
than dropping the stream.
|
||||
- `/api/terminal` accepts a single command line and dispatches to a
|
||||
whitelist (`status`, `isos`, `clients`, `gate {list,assign,release}`,
|
||||
`nfs {list,mount,unmount,scan}`, `smb {status,start,stop,reload}`,
|
||||
`log {clear,tail}`). Output is mirrored onto the LogBus so reading the
|
||||
live tail tells the same story as scrolling the terminal pane.
|
||||
- The whitelist exists deliberately — exposing a raw shell to the web
|
||||
would be an RCE endpoint.
|
||||
|
||||
**Network tab**:
|
||||
- `/api/network` exposes auto-detected `nic_name`, `subnet_mask`, and
|
||||
`gateway` (parsed from `ip route` / `ip addr` at startup). These are
|
||||
read-only by design — silently changing the public IP on a hot UI
|
||||
would break PXE for every client mid-boot.
|
||||
- The only writable network field is `dns_server`, an optional
|
||||
informational hint stored in `Settings`. PXEForge does not run a DNS
|
||||
server; the field exists so operators don't have to dig out the
|
||||
upstream DNS at 3 AM.
|
||||
|
||||
**Bootimus parity nicks** (Bootimus v0.1.55 → v0.1.62):
|
||||
- Storage table tints amber for ISOs that won't boot with current
|
||||
settings (Windows ISO when Windows is disabled, Linux ISO with no
|
||||
detected kernel + over the sanboot size threshold). Each row carries
|
||||
an inline reason — same affordance as Bootimus's "Image Properties"
|
||||
warning.
|
||||
- Dashboard surfaces a "Images that won't boot" panel reusing the same
|
||||
predicate, so the operator sees the problem before they pick the ISO
|
||||
in the gate.
|
||||
- Streaming uploads are already in place via axum multipart; the v0.1.62
|
||||
fix to "502 on big upload" doesn't apply.
|
||||
|
||||
## What's deferred to Phase 5
|
||||
|
||||
- Full ISO9660 + Joliet + Rock Ridge parser (current lookup is plain ISO9660 — Debian ISOs with Rock Ridge extensions may miss some paths).
|
||||
- Real-hardware Windows boot validation (plumbing tested; no MS ISO pushed through the full pipeline yet).
|
||||
- Real-hardware NFS validation (mount manager tested; no real NAS pushed
|
||||
through the full pipeline yet).
|
||||
- UEFI HTTP Boot protocol (option 60 = `HTTPClient`) untested on real firmware.
|
||||
- Raspberry Pi netboot quirks (option 43 vendor-specific, per-MAC prefixes).
|
||||
- Multi-replica / daemonset deployment model (single replica for now).
|
||||
- Pure-Rust SMB server (replace smbd) — slim image, no Samba.
|
||||
- Auto-install / autounattend file library (Bootimus v0.1.58 pattern).
|
||||
- Per-client / per-group menus (Bootimus v0.1.16 pattern).
|
||||
Reference in New Issue
Block a user