# OpenPXE architecture

## Protocol stack

```
Client firmware PXE ROM
        │
        │ DHCPDISCOVER (UDP/67 broadcast, option 60 "PXEClient", option 93 arch)
        ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  OpenPXE                                                               │
│                                                                         │
│    ┌──────────────┐     ┌──────────────┐      ┌──────────────────────┐  │
│    │ DHCP proxy   │     │ TFTP server  │      │  HTTP server (axum)  │  │
│    │ :67, :4011   │     │ :69          │      │  :80                 │  │
│    │  (dhcproto)  │     │  (custom)    │      │                      │  │
│    └──────┬───────┘     └──────┬───────┘      └──────┬───────────────┘  │
│           │                    │                     │                  │
│           └────────────────────┼─────────────────────┘                  │
│                                │                                        │
│                     ┌──────────▼────────────┐                           │
│                     │  IsoStore (on disk)   │                           │
│                     │  + ClientRegistry     │                           │
│                     └───────────────────────┘                           │
└─────────────────────────────────────────────────────────────────────────┘
        │
        ▼ reply with option 60, option 66 tftp-server, option 67 bootfile
Client firmware PXE ROM
        │
        │ TFTP RRQ: snponly.efi (or undionly.kpxe for BIOS)
        ▼
Client runs iPXE
        │
        │ DHCPDISCOVER with option 77 "iPXE"
        ▼
OpenPXE sees user-class "iPXE" → replies with HTTP URL: /boot.ipxe
        │
        │ HTTP GET /boot.ipxe  (iPXE menu, auto-generated from IsoStore)
        ▼
User picks entry; iPXE chains /boot/<id>.ipxe
        │
        │ HTTP GET kernel + initrd (or wimboot + WIM files)
        ▼
Kernel boots with distro-specific args pointing back at /iso/<id>.iso
```

## Crate layout

| Crate               | Responsibility                                                    |
|---------------------|-------------------------------------------------------------------|
| `openpxe-core`     | Shared types: `Config`, `ClientArch`, `FirmwareClass`, `ClientRegistry` |
| `openpxe-ipxe-assets` | Embeds bundled iPXE binaries via `rust-embed`                 |
| `openpxe-iso-store`   | On-disk ISO store, introspection, boot-entry generation        |
| `openpxe-dhcp-proxy`  | UDP listener + `dhcproto` reply builder; pure `decide()` unit-testable |
| `openpxe-tftp`        | RFC 1350 + OACK (blksize / tsize / windowsize). Serves only embedded assets — no filesystem |
| `openpxe-http-api`    | `axum` router: web UI, API, iPXE script generation, ISO streaming |
| `openpxe-webui`       | Single `index.html` served as static string                    |
| `openpxe` (bin)       | Wires everything together, runs the three servers concurrently |

## Key decisions and why

### DHCP proxy only, not a full DHCP server

Proxy mode (RFC 4578) replies with boot parameters (`siaddr`, option 66, 67)
but never sets `yiaddr` — it does not lease IPs. The client merges proxy
replies with its normal DHCP lease from the network's existing DHCP server.

This sidesteps needing `CAP_NET_RAW` or AF_PACKET. A full DHCP server has
to craft Ethernet frames to a client that doesn't yet have an IP — that
requires raw sockets and elevated privileges. A proxy replies on UDP to a
client that already has an IP (or will shortly, from the other DHCP server),
so plain `SOCK_DGRAM` is enough.

### Two-stage iPXE chain

1. Firmware PXE ROM sends DHCPDISCOVER with option 60 = `PXEClient`,
   option 93 = arch.
2. OpenPXE replies with TFTP server + arch-specific iPXE binary
   (`undionly.kpxe` for Legacy BIOS, `snponly.efi` for x86_64 UEFI, etc.).
3. Client TFTPs the iPXE binary and runs it.
4. iPXE does its own DHCP, setting option 77 (user-class) to `iPXE`.
5. OpenPXE detects the user-class and this time replies with an HTTP URL
   in option 67 pointing at `/boot.ipxe`.
6. iPXE fetches and executes that script, which chains the selected OS.

The split is essential because firmware PXE ROMs only speak TFTP; they
don't do HTTP. iPXE adds HTTP (and a lot more), and is small enough to fit
in the TFTP hop.

### TFTP option negotiation is mandatory

Default 512-byte blocks means an ~1 MiB iPXE binary is ~2000 packets.
On anything less than pristine wired Ethernet this is unusably slow or
outright fails. We negotiate:

- `blksize` (RFC 2348): up to ~1468 bytes for Ethernet MTU
- `tsize` (RFC 2349): file size; some PXE ROMs require it present
- `windowsize` (RFC 7440): 8–16 gives order-of-magnitude throughput gains

### Do not touch the client OS trust store

Whatever we do for Windows, we never:

- ship drivers signed with test/development certificates
- instruct users to enable `bcdedit /set testsigning on`
- install any certificate into the target's root/trust store

iVentoy's `httpdisk.sys` approach broke this rule. OpenPXE doesn't.

### Linux ISO boot uses kernel+initrd extraction, not sanboot

Loading a full ISO into RAM via `memdisk` or `sanboot` fails:
- Above ~1–2 GiB, RAM emulation is too slow or too large.
- Most modern distros can't find the emulated CD device from the initramfs.

We instead extract `vmlinuz` + `initrd` at upload time (well, look up their
locations — we serve them via ISO9660 byte-range lookup, no extraction on
disk) and pass distro-specific kernel args that point the installer back at
the HTTP-served ISO.

### No iPXE UX

iPXE is an implementation detail. The web UI only accepts `.iso` uploads
and shows distros, not `.ipxe` scripts or boot targets. If someone who
knows iPXE wants to peek, they can `curl /boot.ipxe` — that's fine. But
the UI never surfaces it.

## Unit tests

Run with `cargo test --workspace --lib`. Current coverage:

- `ClientArch` alias handling (0x0009 → x86_64 UEFI)
- `FirmwareClass` classification (iPXE wins over PXEClient echo)
- Raw option-93 extraction from the wire
- TFTP RRQ parsing with options
- HTTP Range header parsing (full, open-ended, suffix, explicit)
- Distro family detection from volume label
- ISO id slugification

## Phase 3 hardening

A code-review pass after Phase 2 turned up 16 issues across the protocol
stack and container surface. Everything P0/P1/P2 is addressed in this
release:

**Protocol reliability** (review P1 #10/#12):
- TFTP retransmit loop rewritten with explicit window tracking so UEFI SNP
  clients no longer hang on files that end mid-window. Covered by 4 new
  unit tests (`plan_window_*` in `crates/tftp`).
- DHCP proxy now honors the broadcast-flag bit (RFC 2131 §4.1) when
  choosing unicast vs broadcast reply destination.

**Container posture** (review P1 #2/#13/#14/#15):
- Multi-arch container (linux/amd64 + linux/arm64) via `buildx`.
- New entrypoint (`deploy/docker/entrypoint.sh`) chowns data dirs as root,
  then drops to uid 10001 via `gosu` — fixes the "bind-mount comes up
  root-owned" problem that breaks ISO upload on standard Docker hosts.
- `/healthz` and `/readyz` split from `/api/status` — readyz fails if no
  iPXE binaries are bundled, giving K8s probes a real signal.
- Startup aborts with a clear error if `OPENPXE_PUBLIC_IP` can't be
  auto-detected (no more silent `127.0.0.1` advertisement).
- `scripts/fetch-ipxe.sh` fails non-zero if zero binaries download; the
  Dockerfile uses arch-scoped paths (`x86_64-efi/snponly.efi` etc.).

**Windows boot plumbing** (new):
- `openpxe-iso-store::smb::SmbManager` supervises `smbd` on the Windows
  toggle: `start` → spawn + write `smb.conf`; `reconcile` → SIGHUP on
  share changes; `stop` → SIGTERM. `SmbState` surfaced to the UI for
  visibility.
- `extract_windows_iso` shells out to `7z` (or `bsdtar` fallback) to
  unpack the ISO tree into `smb_dir/<slug>/` as the SMB share root.
- `WimPatcher` (from Phase 2) is still in place for `boot.wim` injection.
- Guardrail: flipping the Windows toggle in `/api/settings` is rejected
  with a 400 if `wimboot` isn't bundled.

**iVentoy/Bootimus parity** (P2):
- ISO sizes in menu labels (`[ 4376 MB]`), iVentoy format.
- `Reboot Computer` + `Exit and continue BIOS boot` in Tools menu.
- Number-key hotkeys (1..9) on boot entries, letter hotkeys on tools.
- Clients tab cross-joins the deployment queue so an operator sees "in queue #2"
  or "assigned: ubuntu-linux" status inline.

**Developer ergonomics** (new):
- `openpxe seed --from <path>` CLI to import ISOs from a directory.
  Same pipeline as web upload (slug, sha256, introspection, boot-entry).
- `docker-compose.yml` with `openpxe` (host network, real PXE) and
  `openpxe-dev` (published ports, DHCP disabled, for API testing).

**API cleanliness**:
- All timestamps now serialized as RFC 3339 strings (the `time` crate's
  default 9-tuple broke browser `Date` parsing).
- Queue poll retains assignment until the operator releases it; if the
  client's chain fails, it reuses the assignment instead of falling back
  to the menu.

## Phase 4 — UI restructure + remote storage

The web UI was rebuilt around six tabs (Dashboard / Network / Queue /
Storage / Terminal / About) inspired by the iVentoy layout the user
attached and Netbox Labs's compact-card pattern. The old hierarchical
"Monitoring / Content / Configuration" sidebar grouping is gone — every
tab is one click from the brand bar.

**NFS share manager** (`crates/iso-store/src/nfs.rs`):
- Operators add a remote share via Storage → NFS shares; OpenPXE mounts
  it under `<work_dir>/nfs/<id>/` and walks it for `*.iso` files.
- Each ISO found is registered with `IsoStore::register_external` using
  a new `IsoSource::Nfs { mount_id, relative_path }` variant. The store
  resolves these to disk lazily via `set_nfs_root`, so adding NFS
  required exactly one new field on `IsoMeta` (with a serde default for
  forward compatibility with old `meta.json` files).
- Versions: `vers=3` and `vers=4.1` only. v3 also gets `nolock` since
  appliances commonly disable `lockd`. All mounts use `soft,timeo=100`
  so a dead server surfaces as a UI error rather than wedging iPXE.
- State persists to `<work_dir>/nfs.json`. On startup the manager
  re-attempts every spec; failures are logged per-mount and surfaced in
  the UI rather than blocking startup.
- Container requirement: `mount.nfs` binary plus `CAP_SYS_ADMIN`. The
  default Dockerfile bundles `nfs-common`. OpenShift operators must
  swap to a more permissive SCC or use a CSI driver.

**Live log + operator terminal** (`crates/core/src/log_bus.rs`,
`crates/http-api/src/{log_stream,terminal}.rs`):
- A `LogBusLayer` feeds every tracing event into a 500-line ring buffer
  plus a `tokio::sync::broadcast` channel.
- `/api/log/stream` is an SSE endpoint that emits the recent buffer
  followed by live updates. Slow clients see a `lagged` event rather
  than dropping the stream.
- `/api/terminal` accepts a single command line and dispatches to a
  whitelist (`status`, `isos`, `clients`, `queue {list,assign,release}`,
  `nfs {list,mount,unmount,scan}`, `smb {status,start,stop,reload}`,
  `log {clear,tail}`). Output is mirrored onto the LogBus so reading the
  live tail tells the same story as scrolling the terminal pane.
- The whitelist exists deliberately — exposing a raw shell to the web
  would be an RCE endpoint.

**Network tab**:
- `/api/network` exposes auto-detected `nic_name`, `subnet_mask`, and
  `gateway` (parsed from `ip route` / `ip addr` at startup). These are
  read-only by design — silently changing the public IP on a hot UI
  would break PXE for every client mid-boot.
- The only writable network field is `dns_server`, an optional
  informational hint stored in `Settings`. OpenPXE does not run a DNS
  server; the field exists so operators don't have to dig out the
  upstream DNS at 3 AM.

**Bootimus parity nicks** (Bootimus v0.1.55 → v0.1.62):
- Storage table tints amber for ISOs that won't boot with current
  settings (Windows ISO when Windows is disabled, Linux ISO with no
  detected kernel + over the sanboot size threshold). Each row carries
  an inline reason — same affordance as Bootimus's "Image Properties"
  warning.
- Dashboard surfaces a "Images that won't boot" panel reusing the same
  predicate, so the operator sees the problem before they pick the ISO
  in the queue.
- Streaming uploads are already in place via axum multipart; the v0.1.62
  fix to "502 on big upload" doesn't apply.

## Phase 5 — pre-beta hardening

**Per-MAC host bindings** (`crates/core/src/host_bindings.rs`):
- New `HostBindings` registry maps a MAC → preferred `BootEntry::id`
  (or one of the reserved menu shortcuts (`_local`, `_queue``,
  `_tools_menu`).
- Persisted to `<work_dir>/hosts.json`. Like `SettingsStore`, in-memory
  is authoritative — disk corruption falls back to empty rather than
  failing startup.
- Inspired by Tinkerbell `smee`'s MAC-prepended URL pattern. The DHCP
  reply now embeds `?mac=${mac}` in the boot.ipxe URL; iPXE substitutes
  the literal MAC client-side, so the HTTP layer can short-circuit
  past the menu when a binding exists.
- `/api/hosts` GET / POST / DELETE drives the **Hosts** tab.

**Prometheus metrics** (`crates/core/src/metrics.rs`):
- Lock-free `AtomicU64`-backed counters + gauges. No `prometheus` /
  `metrics-rs` dep — they bring a registry, runtime, and complexity
  we don't need for a fixed set of metric families.
- Counters: DHCP replies (per arch label), DHCP declined, TFTP
  transfers (per status label), TFTP bytes, HTTP requests (per route
  label).
- Gauges: ISO count, client count, queue count, queue-imaging count,
  NFS active mounts, uptime, build info.
- Exposed as plain Prometheus text at `/metrics`.

**Code cleanup pass**: clippy `--workspace --all-targets` is now
warning-free. Replaced `format!()`-into-`String` with
`std::fmt::Write::write!`, switched manual reverse comparators to
`Reverse`, fixed `map_or(false, …)` → `is_some_and`, and a handful of
other idiom fixes.

**UI overhaul** for the pre-beta milestone:
- Light + dark themes via `:root[data-theme=light]` token swap.
  Toggled by a top-right button or the `T` key. Persisted in
  localStorage; pre-paint inline script avoids dark→light flash.
- New SVG logos: a refined OpenPXE mark (`logo.svg`) and a compact
  SMIL-animated loader (`loader.svg`). Pure SVG, embedded in the binary.
- Deployment progress widget on the Dashboard and Queue: animated
  OpenPXE mark paired with a `linear-gradient(warn → accent)` progress bar
  with a moving sheen. Goes idle (greyscale, no sheen) at zero
  imaging load.
- Loader replaced "Loading..." text with the same OpenPXE mark.
- Sidebar gains a **Hosts** tab.

**Windows boot validation**:
- New integration test synthesizes an ISO9660 with the `SOURCES\BOOT.WIM`
  sentinel, uploads it, and asserts:
  1. introspection labels it `windows_pe` with `has_boot_wim=true`,
  2. the boot entry is `BootKind::Wimboot` with all five canonical
     files (`bootmgr`, `bootmgr.efi`, `bcd`, `boot.sdi`, `boot.wim`),
  3. the rendered iPXE script chains wimboot with `initrd --name`
     entries for each, and
  4. **no** trust-store strings appear: `bcdedit`, `testsigning`,
     `certutil`, `httpdisk`, `test-signed` are all explicitly
     forbidden in the rendered output.
- WinPE bootstrap (`startnet.cmd`) now picks up Bootimus v0.1.58
  fixes: explicit `net start Workstation` before `net use`, surfaces
  errors instead of blind retries.

**Test posture**: protocol, HTTP, ISO-store, Windows script, queue, metrics,
and UI-offline checks all run in the workspace test suite.

## What's deferred to Phase 6

- Full ISO9660 + Joliet + Rock Ridge parser (current lookup is plain ISO9660 — Debian ISOs with Rock Ridge extensions may miss some paths).
- Real-hardware Windows boot validation (plumbing tested; no MS ISO pushed through the full pipeline yet).
- Real-hardware NFS validation (mount manager tested; no real NAS pushed
  through the full pipeline yet).
- UEFI HTTP Boot protocol (option 60 = `HTTPClient`) untested on real firmware.
- Raspberry Pi netboot quirks (option 43 vendor-specific, per-MAC prefixes).
- Multi-replica / daemonset deployment model (single replica for now).
- Pure-Rust SMB server (replace smbd) — slim image, no Samba.
- Auto-install / autounattend file library (Bootimus v0.1.58 pattern).
- Per-client / per-group menus (Bootimus v0.1.16 pattern).
- Real-hardware integration: at minimum a Linux ISO booted on a real
  BIOS box, a Windows ISO booted via wimboot on a real UEFI box, and a
  Pi 4 booting from an NFS-mounted Raspberry Pi OS ISO.
- Distro profile manifest (Bootimus v0.1.27 pattern) — currently
  introspection logic is hard-coded; could become data-driven so an
  operator can add a new distro profile from the UI without rebuilding.
- Wake-on-LAN trigger (Bootimus v0.1.16 pattern) — power-on a host then
  imaging starts unattended via a per-MAC binding.
- Syslog receiver (smee feature) — capture client-side install syslog
  for diagnostic visibility.
- IPv6 PXE / DHCPv6 — currently IPv4 only.