Compare commits

..
7 Commits
Author SHA1 Message Date
Miles WardandClaude Opus 4.8 1c262a6d61 v0.8.0: dep prune, memtest introspection fix, concurrent uploads, x-api-key
Dependency cleanup (ponytail audit):
- Drop 14 unused dependency declarations across 7 crates; quick-xml and
  x509-parser leave the tree entirely (SAML cert/XML work is handled by
  bergshamra + roxmltree).

Fixes:
- introspect: drop the over-broad "microsoft" UTF-16 bulk-scan marker that
  mislabeled Secure-Boot-signed non-Windows bootables (memtest86, signed
  BSDs, firmware tools) as Windows — the string lives in their MS-signed
  EFI loader's FAT long-filename entries. INTROSPECT_REV 3 -> 4 re-probes
  existing local ISOs on startup so the bogus label clears on upgrade.
- upload: begin_upload now reclaims an abandoned <id>.partial instead of
  rejecting the re-upload with "already uploading". Robust against browser
  refresh, tab close, and dropped connections (the chunked protocol can't
  resume a dead session anyway).

Features:
- Storage upload: multi-file + concurrent. Each dropped/selected .iso gets
  its own progress row and uploads independently; a single page-leave guard
  plus a pagehide keepalive-abort replace the old shared singletons.
- Operator API key (x-api-key): a persisted key authenticates /api/* exactly
  like an operator session, for Postman/scripts. New core ApiKeyStore
  (generated on first run, regenerable), accepted in require_auth alongside
  the session cookie, surfaced in Settings -> Advanced with copy + regenerate
  and a usage reference. GET /api/api-key + POST /api/api-key/regenerate.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-21 23:35:42 -04:00
Miles WardandClaude Opus 4.8 27703c437a docs(compose): simplify root docker-compose to one pull-based service
The root compose was scratch-built and carried both a prod and a dev
service plus a local build stanza. Replace it with a single
deployment-focused service that pulls
gitea.milesward.dev/mward4/openpxe:latest (matching the Unraid
template and the homelab flow):

- drop the openpxe-dev service and the build: context (deploy, not build)
- HTTP on 4200 so it clears an Unraid webGUI / reverse proxy on :80
- host networking (DHCPDISCOVER is broadcast — bridges don't forward it)
- cap_add NET_BIND_SERVICE instead of privileged; the binary already
  carries cap_net_bind_service as a file capability
- OPENPXE_PUBLIC_IP stays a required, fail-fast variable
- isos + work bind mounts (SMB dir omitted — Windows boots via HTTP
  sanboot since v0.5.8, no SMB server needed)

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-14 08:15:43 -04:00
Miles WardandClaude Opus 4.8 1ded291c7b v0.7.5: Joliet namespace fallback, gap-tolerant El Torito walk, Unattended pagination
Follow-up to the v0.7.4 dashboard triage: VCSA/ultravnc data ISOs are
*correctly* flagged non-bootable (no boot catalog exists to find), but
two real false-negative holes could mislabel genuinely bootable
appliance ISOs — both closed here.

iso_fs:
- Joliet fallback (the big one): lookup() now tries the primary
  ISO9660 namespace first and falls back to the Joliet SVD (UCS-2
  big-endian identifiers, escape-sequence detected). Windows-oriented
  mastering tools — common for vendor/appliance ISOs — write a minimal
  or mangled primary tree and keep the real filenames only in Joliet;
  those images probed as "no installer files" and their in-ISO fetches
  404'd. Applies everywhere the walker is used: introspection probes
  (local + NFS/SFTP) and /iso/{id}/{*path} serving.
- find_descriptor(): the PVD/SVD search scans the whole descriptor
  area (LBA 16..32), skipping non-CD001 filler sectors instead of
  requiring a pristine sector 16.
- TestIsoBuilder grows a joliet_only mode (bare primary tree, real
  names only in the SVD) modeling the mastering worst case.

introspect:
- detect_el_torito() no longer aborts at the first non-CD001 sector or
  stops at a Set Terminator — sloppy mastering leaves zeroed filler
  sectors that used to hide a real boot record and flag a bootable
  image as a data ISO. All 16 descriptor sectors are examined; the
  25-byte exact signature can't false-positive on what follows the set.
- Volume-label read now uses the same tolerant descriptor scan, and
  label + El Torito + namespace probes all share one CachingReadAt, so
  remote probes spend fewer round-trips than before despite scanning
  more sectors.
- INTROSPECT_REV bumped to 3 so everything probed by the rev-2 logic
  re-probes with the Joliet fallback: local ISOs on first startup, and
  remote ISOs via the rev-gated cache self-invalidating.

webui:
- Unattended files: the same 5-per-page pager as Available images
  (Showing X–Y of N · Prev/Next), composed with the existing filter,
  page resets on input.

Validation: clippy pedantic clean, fmt clean, 319 workspace tests
green (+3: joliet fallback lookup, joliet-only classification, filler-
sector boot record), webui syntax-checked.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-12 17:04:01 -04:00
Miles WardandClaude Opus 4.8 6524aa4118 v0.7.4: probe-based introspection — remote shares classify, gparted bug fixed, Storage pagination
Introspection (the headline): detection is now probe-based. Instead of
grepping raw sectors for filename strings, we walk the ISO9660
directory tree and check whether the well-known boot files actually
exist — and the same probes run over NFS READ3 / SFTP seek-reads, so
share-hosted ISOs finally classify instead of registering as Unknown.

iso-store:
- New iso_fs module: the read-only ISO9660 walker (generalized from
  http-api) over an IsoReadAt trait — local files, NFS, SFTP, and the
  in-memory test images all share it. Iterative walk, 4 MiB directory
  cap, strict-mastering trailing-dot normalization (VMLINUZ.;1 now
  matches /vmlinuz), CachingReadAt collapses repeated directory reads
  during the probe pass (~60 → ~6 round-trips per remote ISO).
- introspect.rs rewritten (INTROSPECT_REV 2): PVD label → El Torito →
  /sources/boot.wim probe → verified Linux kernel+initrd probe table →
  local-only 16 MiB UDF-Windows scan → filename-token fallback.
  * Fixes the false-Windows bug: any Linux ISO shipping GRUB/syslinux
    chainload modules contains the literal "bootmgr", so gparted-live
    classified as WindowsPe. Linux probes now run first; the byte scan
    only sees ISOs nothing else claimed. Local ISOs re-probe once on
    startup via the rev bump — no re-upload.
  * Kernel entries are emitted only when kernel+initrd verifiably
    exist (no more guessed paths that 404 at boot). Debian-live /
    d-i netinst / CoreOS shapes classify for the UI but keep their
    working sanboot entries (their boot protocols need args we don't
    render yet; CoreOS additionally needs its embedded ignition).
  * Label + filename vocab extended: rhcos/coreos/openshift/okd,
    gparted/clonezilla/kali/tails, almalinux/rocky, sles, manjaro.
- NFS + SFTP managers: per-ISO IsoReadAt readers (READ3-at-offset with
  short-read looping / seek+read_exact), background introspection pass
  after each scan — entries register instantly with a provisional
  filename-based report (rev 0, optimistic sanboot preserved) and
  upgrade in place as probes land (30s/ISO timeout, failures keep the
  provisional). locate_in_iso() exposes the walker to the HTTP layer.
- remote_cache: introspection results persisted per protocol keyed
  share/path@size and gated on INTROSPECT_REV — container restarts
  re-probe only new/replaced ISOs; upgrades re-probe exactly once.
- SMB: smbclient can't seek, so SMB ISOs get the filename-token family
  (rev stays 0 → sanboot entry + "awaiting introspection" label).
- IsoStore::update_external_introspection swaps in completed reports
  and regenerates boot entries, preserving category/password.

http-api:
- /iso/{id}/{*path} now serves files from inside NFS/SFTP-hosted ISOs
  (remote ISO9660 lookup + ranged share stream) — verified kernel
  entries on remote Linux ISOs are actually bootable, end to end.
- iso_fs.rs deleted in favor of the shared iso-store module.
- full_flow fixtures build real directory trees via the shared
  test-image builder (new iso-store feature) — a label-only blob no
  longer earns a kernel entry, by design.

webui:
- Available images: paged 5 per page with a quiet footer pager
  (Showing X–Y of N · Prev/Next), filter-then-paginate, page resets on
  search input. Fifty images is five clean pages, not a scroll wall.
- Hosts/Queue profile: "Unattended file (in Storage → Advanced)" so
  the picker says where the files live.
- Row badge keys on introspect_rev: probed remote ISOs read like local
  ones; un-probed say "awaiting introspection".

Validation: clippy pedantic clean, fmt clean, 316 workspace tests
green (+17: walker, probe shapes incl. gparted regression + CoreOS,
filename table, cache round-trips), webui syntax-checked.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-12 15:21:14 -04:00
Miles WardandClaude Opus 4.8 934cfbab46 v0.7.3: UI cleanup — aligned Hosts pin form, native-chrome list filters, Link row reorder
Design-language cleanup of the v0.7.2 additions (no behaviour change).

Hosts:
- The Pin MAC form collapses to a single aligned 4-up row: MAC ·
  Label · Architecture · Boot binary. Target drops full-width onto its
  own line beneath them. The per-field hints that broke the row's
  alignment moved into the explanatory note below, so every control
  shares one baseline.

Storage:
- The image and unattended filter inputs are now wrapped in a
  label.field, so they inherit the standard text-field chrome (border,
  radius, height, focus ring) instead of the raw browser
  <input type=search> look. They span the full card width for
  continuity with the rest of the page.

Network:
- The 'Link' row moved below 'Public base URL'. Its joined
  operstate · speed · duplex · MAC string runs long, so placing it last
  lets it wrap at the bottom without shoving the other rows around.

Validation: clippy clean, fmt clean, 299 workspace tests green, webui
syntax-checked. No protocol or boot-path changes in this release.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-09 22:44:19 -04:00
Miles WardandClaude Opus 4.8 a71057fce6 v0.7.2: UI polish — list search, unified Hosts form, NIC link details, About refresh, spacing
Storage:
- Filter inputs for the Available images table (matches filename,
  detected family, category, source) and the Unattended files list
  (name, kind). Pure client-side; shown when there's more than one
  entry. Forty-image libraries are now navigable.

Hosts — one form, one mental model:
- The separate Boot rules card is gone. The Pin form gains
  'Architecture (optional)' next to Label (plus the v0.7.1 boot-binary
  pin): a full MAC with no architecture saves a per-host pin exactly as
  before; a MAC prefix and/or architecture saves a first-match-wins
  group rule. Saved rules render as a compact read-only 'Group rules'
  card with remove buttons.
- The boot-decision webhook keeps working via /api/boot-rules but no
  longer has a UI knob (operator feedback: not needed in the UI).
- Per-machine auto-deploy fields are rejected on group rules with a
  clear message (they're per-host values).

Network:
- New 'Link' row under NIC name: operstate · speed · duplex · port MAC,
  read from sysfs at startup (detect_link_info). Empty-degrades on
  non-Linux dev builds and virtual NICs. Confirms WHICH physical port
  answers PXE in multi-NIC/trunked environments.

About:
- Hero copy rewritten: positioning lead, three-pillar feature grid
  (Boot anything / Adapt to every machine / Run it in production), and
  the privacy + no-test-cert principles restated crisply.

Spacing:
- label.field:has(+ button) collapse fixes the doubled 30px gap above
  Queue 'Launch for all waiting' and Network 'Save' (now the same 16px
  as every other card action).

Validation: clippy clean, fmt clean, 299 workspace tests green, webui
syntax-checked. No protocol or boot-path changes in this release.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-09 21:47:48 -04:00
Miles WardandClaude Opus 4.8 29040e8a5a v0.7.1: walk the ladder once ever — persistent learned modes, rule pins, same-boot iPXE recovery
Answers the operational question 'can a machine try all three boot
binaries in one go?' The protocol can't carry three NBPs in one cycle
(one boot file per DHCP round, the Secure-Boot refusal happens after
handoff with no error report, and the broken-NIC case specifically needs
the firmware itself to load builtin-driver iPXE — GRUB's network rides
the same broken firmware stack). What we CAN do is make the walk a
once-per-machine-ever event and give operators a way to skip it:

- Learned driver modes persist (<work_dir>/driver_modes.json). A MAC
  that reaches the Shim rung, or confirms an iPXE handoff at Builtin,
  is pinned to disk: immune to the 30-min TTL, reloaded at startup.
  The file only carries exceptions — a healthy fleet never writes it.
  Corrupt file starts empty (standard crash-cache policy).
- Boot rules gain an optional driver_mode pin (auto/firmware/builtin/
  shim), consulted by the DHCP proxy BEFORE the escalation ladder:
  'this OUI is a Secure Boot rack -> serve shim immediately' = zero
  failed cycles. Mode-only rules coexist with target rules (a pin
  doesn't shadow a later target match). Editor column on Hosts tab.
- grub.cfg now tries to chainload all-drivers iPXE before showing the
  signed menu: with SB off the chainload succeeds and the client gets
  the full iPXE feature set back in the SAME boot (self-healing for
  mis-escalations, and the handoff then pins the working mode); with
  SB on, shim's verifier refuses it inline — no reboot — and the
  signed menu appears.

DhcpProxyServer now takes the escalation table + rules store from main
(persistence path comes from the configured work dir).

Validation: clippy clean, fmt clean, 299 workspace tests green (+9:
persistence round-trip across restart, Shim pin survives TTL, learned
Builtin survives TTL, corrupt-file recovery, default-mode-never-
persisted, rule-pin matching incl. unknown-mode tolerance and
pin/target coexistence, GRUB chainload-before-menu ordering, API
round-trip of the driver_mode field).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-09 21:16:22 -04:00
33 changed files with 3319 additions and 868 deletions
Generated
+11 -136
View File
@@ -175,45 +175,6 @@ dependencies = [
"password-hash", "password-hash",
] ]
[[package]]
name = "asn1-rs"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7f43a50ac4fdca5df8e885c21b835997f0a1cdee65494a6847694a98652d9d8"
dependencies = [
"asn1-rs-derive",
"asn1-rs-impl",
"displaydoc",
"nom 7.1.3",
"num-traits",
"rusticata-macros",
"thiserror",
"time",
]
[[package]]
name = "asn1-rs-derive"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c"
dependencies = [
"proc-macro2",
"quote",
"syn",
"synstructure",
]
[[package]]
name = "asn1-rs-impl"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b18050c2cd6fe86c3a76584ef5e0baf286d038cda203eb6223df2cc413565f7"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]] [[package]]
name = "assert-json-diff" name = "assert-json-diff"
version = "2.0.2" version = "2.0.2"
@@ -1144,20 +1105,6 @@ dependencies = [
"zeroize", "zeroize",
] ]
[[package]]
name = "der-parser"
version = "10.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6"
dependencies = [
"asn1-rs",
"displaydoc",
"nom 7.1.3",
"num-bigint",
"num-traits",
"rusticata-macros",
]
[[package]] [[package]]
name = "der_derive" name = "der_derive"
version = "0.7.3" version = "0.7.3"
@@ -2445,7 +2392,7 @@ dependencies = [
"httpdate", "httpdate",
"idna", "idna",
"mime", "mime",
"nom 8.0.0", "nom",
"percent-encoding", "percent-encoding",
"quoted_printable", "quoted_printable",
"rustls", "rustls",
@@ -2564,12 +2511,6 @@ dependencies = [
"unicase", "unicase",
] ]
[[package]]
name = "minimal-lexical"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
[[package]] [[package]]
name = "miniz_oxide" name = "miniz_oxide"
version = "0.8.9" version = "0.8.9"
@@ -2702,16 +2643,6 @@ dependencies = [
"libc", "libc",
] ]
[[package]]
name = "nom"
version = "7.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
dependencies = [
"memchr",
"minimal-lexical",
]
[[package]] [[package]]
name = "nom" name = "nom"
version = "8.0.0" version = "8.0.0"
@@ -2803,15 +2734,6 @@ dependencies = [
"libc", "libc",
] ]
[[package]]
name = "oid-registry"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12f40cff3dde1b6087cc5d5f5d4d65712f34016a03ed60e9c08dcc392736b5b7"
dependencies = [
"asn1-rs",
]
[[package]] [[package]]
name = "once_cell" name = "once_cell"
version = "1.21.4" version = "1.21.4"
@@ -2836,7 +2758,7 @@ checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
[[package]] [[package]]
name = "openpxe" name = "openpxe"
version = "0.7.0" version = "0.8.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"axum", "axum",
@@ -2848,17 +2770,15 @@ dependencies = [
"openpxe-ipxe-assets", "openpxe-ipxe-assets",
"openpxe-iso-store", "openpxe-iso-store",
"openpxe-tftp", "openpxe-tftp",
"serde",
"time", "time",
"tokio", "tokio",
"toml",
"tracing", "tracing",
"tracing-subscriber", "tracing-subscriber",
] ]
[[package]] [[package]]
name = "openpxe-core" name = "openpxe-core"
version = "0.7.0" version = "0.8.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"base64", "base64",
@@ -2867,7 +2787,6 @@ dependencies = [
"figment", "figment",
"flate2", "flate2",
"parking_lot", "parking_lot",
"quick-xml",
"rcgen", "rcgen",
"roxmltree", "roxmltree",
"serde", "serde",
@@ -2880,27 +2799,26 @@ dependencies = [
"tracing", "tracing",
"tracing-subscriber", "tracing-subscriber",
"uuid", "uuid",
"x509-parser",
] ]
[[package]] [[package]]
name = "openpxe-dhcp-proxy" name = "openpxe-dhcp-proxy"
version = "0.7.0" version = "0.8.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"bytes",
"dhcproto", "dhcproto",
"openpxe-core", "openpxe-core",
"parking_lot", "parking_lot",
"serde_json",
"socket2", "socket2",
"thiserror", "tempfile",
"tokio", "tokio",
"tracing", "tracing",
] ]
[[package]] [[package]]
name = "openpxe-http-api" name = "openpxe-http-api"
version = "0.7.0" version = "0.8.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"axum", "axum",
@@ -2908,7 +2826,6 @@ dependencies = [
"bergshamra", "bergshamra",
"bytes", "bytes",
"futures", "futures",
"hyper",
"image", "image",
"insta", "insta",
"lettre", "lettre",
@@ -2922,7 +2839,6 @@ dependencies = [
"serde", "serde",
"serde_json", "serde_json",
"tempfile", "tempfile",
"thiserror",
"time", "time",
"tokio", "tokio",
"tokio-stream", "tokio-stream",
@@ -2936,17 +2852,16 @@ dependencies = [
[[package]] [[package]]
name = "openpxe-ipxe-assets" name = "openpxe-ipxe-assets"
version = "0.7.0" version = "0.8.0"
dependencies = [ dependencies = [
"openpxe-core", "openpxe-core",
"rust-embed", "rust-embed",
"thiserror",
"tracing", "tracing",
] ]
[[package]] [[package]]
name = "openpxe-iso-store" name = "openpxe-iso-store"
version = "0.7.0" version = "0.8.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"bcrypt", "bcrypt",
@@ -2965,31 +2880,26 @@ dependencies = [
"serde_json", "serde_json",
"sha2 0.10.9", "sha2 0.10.9",
"tempfile", "tempfile",
"thiserror",
"time", "time",
"tokio", "tokio",
"tokio-util",
"tracing", "tracing",
"uuid",
] ]
[[package]] [[package]]
name = "openpxe-tftp" name = "openpxe-tftp"
version = "0.7.0" version = "0.8.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"bytes",
"openpxe-core", "openpxe-core",
"openpxe-ipxe-assets", "openpxe-ipxe-assets",
"socket2", "socket2",
"thiserror",
"tokio", "tokio",
"tracing", "tracing",
] ]
[[package]] [[package]]
name = "openpxe-webui" name = "openpxe-webui"
version = "0.7.0" version = "0.8.0"
[[package]] [[package]]
name = "p256" name = "p256"
@@ -3436,15 +3346,6 @@ version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3" checksum = "a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3"
[[package]]
name = "quick-xml"
version = "0.40.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2474bd2e5029e7ccb6abb2ba48cf2383a333851dedf495901544281590c7da7f"
dependencies = [
"memchr",
]
[[package]] [[package]]
name = "quinn" name = "quinn"
version = "0.11.9" version = "0.11.9"
@@ -3931,15 +3832,6 @@ dependencies = [
"semver", "semver",
] ]
[[package]]
name = "rusticata-macros"
version = "4.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf0c4a6ece9950b9abdb62b1cfcf2a68b3b67a10ba445b3bb85be2a293d0632"
dependencies = [
"nom 7.1.3",
]
[[package]] [[package]]
name = "rustix" name = "rustix"
version = "1.1.4" version = "1.1.4"
@@ -5642,23 +5534,6 @@ dependencies = [
"tls_codec", "tls_codec",
] ]
[[package]]
name = "x509-parser"
version = "0.18.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d43b0f71ce057da06bc0851b23ee24f3f86190b07203dd8f567d0b706a185202"
dependencies = [
"asn1-rs",
"data-encoding",
"der-parser",
"lazy_static",
"nom 7.1.3",
"oid-registry",
"rusticata-macros",
"thiserror",
"time",
]
[[package]] [[package]]
name = "yansi" name = "yansi"
version = "1.0.1" version = "1.0.1"
+1 -3
View File
@@ -12,7 +12,7 @@ members = [
] ]
[workspace.package] [workspace.package]
version = "0.7.0" version = "0.8.0"
edition = "2021" edition = "2021"
rust-version = "1.95" rust-version = "1.95"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
@@ -82,8 +82,6 @@ lettre = { version = "0.11", default-features = false, features = ["smtp-transpo
# SP layer (AuthnRequest, metadata parse, SAMLResponse semantics) on top. # SP layer (AuthnRequest, metadata parse, SAMLResponse semantics) on top.
bergshamra = "0.5" bergshamra = "0.5"
roxmltree = "0.21" roxmltree = "0.21"
quick-xml = "0.40"
x509-parser = "0.18"
# flate2 default backend is miniz_oxide (pure Rust) — do NOT enable the # flate2 default backend is miniz_oxide (pure Rust) — do NOT enable the
# zlib/zlib-ng C backends, which would break the musl-static build. # zlib/zlib-ng C backends, which would break the musl-static build.
flate2 = "1.1" flate2 = "1.1"
+3 -5
View File
@@ -27,13 +27,11 @@ tokio = { workspace = true, features = ["sync", "rt", "macros", "time"] }
bcrypt.workspace = true bcrypt.workspace = true
# v0.5.1: pure-Rust SAML 2.0 SP. bergshamra = XML-DSig verify + exclusive # v0.5.1: pure-Rust SAML 2.0 SP. bergshamra = XML-DSig verify + exclusive
# c14n (no OpenSSL/C). roxmltree/quick-xml parse + build SAML XML; # c14n (no OpenSSL/C), plus IdP signing-cert extraction from metadata.
# x509-parser pulls the IdP signing cert out of metadata; flate2+base64 # roxmltree parses the SAML/metadata XML; flate2+base64 encode the
# encode the HTTP-Redirect binding's SAMLRequest. # HTTP-Redirect binding's SAMLRequest.
bergshamra.workspace = true bergshamra.workspace = true
roxmltree.workspace = true roxmltree.workspace = true
quick-xml.workspace = true
x509-parser.workspace = true
flate2.workspace = true flate2.workspace = true
base64.workspace = true base64.workspace = true
+158
View File
@@ -0,0 +1,158 @@
//! Operator API key — a single persisted secret that authenticates
//! programmatic `/api/*` callers (Postman, scripts, CI) via the
//! `x-api-key` header, as an alternative to the browser session cookie.
//!
//! Generated on first load and persisted to `<work_dir>/api_key.json` so
//! it survives restarts — an operator pastes it into their client once.
//! Regenerable from Settings → Advanced; the previous key stops working
//! the moment a new one is minted. Grants the same access as a logged-in
//! operator (the middleware treats a valid key exactly like a session).
use parking_lot::RwLock;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
use std::sync::Arc;
use uuid::Uuid;
#[derive(Debug, Clone, Serialize, Deserialize)]
struct ApiKeyFile {
key: String,
}
/// Persisted operator API key. Cheap to clone (Arc-shared); contention is
/// nil (read on every authenticated request, written only on regenerate).
#[derive(Debug, Clone)]
pub struct ApiKeyStore {
path: Arc<PathBuf>,
inner: Arc<RwLock<String>>,
}
impl ApiKeyStore {
/// Load the stored key, minting + persisting a fresh one on first run
/// (or when the file is missing / corrupt / empty).
#[must_use]
pub fn load_or_init(work_dir: &std::path::Path) -> Self {
let path = work_dir.join("api_key.json");
let key = match std::fs::read_to_string(&path) {
Ok(text) => serde_json::from_str::<ApiKeyFile>(&text)
.map(|f| f.key)
.ok()
.filter(|k| !k.is_empty())
.unwrap_or_else(generate_key),
Err(_) => generate_key(),
};
let store = Self {
path: Arc::new(path),
inner: Arc::new(RwLock::new(key)),
};
// Land a first-run (or repaired) key on disk immediately so it's
// stable across the very next restart.
store.persist();
store
}
#[must_use]
pub fn current(&self) -> String {
self.inner.read().clone()
}
/// Constant-time comparison against the stored key. An empty candidate
/// never matches, so a blank/absent header can't authenticate.
#[must_use]
pub fn verify(&self, candidate: &str) -> bool {
if candidate.is_empty() {
return false;
}
ct_eq(self.inner.read().as_bytes(), candidate.as_bytes())
}
/// Mint a fresh key, persist it, and return it. The previous key is
/// invalid the instant this returns.
#[must_use]
pub fn regenerate(&self) -> String {
let key = generate_key();
self.inner.write().clone_from(&key);
self.persist();
tracing::info!(target: "openpxe::auth", "operator API key regenerated");
key
}
fn persist(&self) {
let body = match serde_json::to_vec_pretty(&ApiKeyFile {
key: self.current(),
}) {
Ok(b) => b,
Err(e) => {
tracing::warn!(target: "openpxe::auth", "serialize api_key.json: {e}");
return;
}
};
if let Some(parent) = self.path.parent() {
let _ = std::fs::create_dir_all(parent);
}
let tmp = self.path.with_extension("json.tmp");
if let Err(e) = std::fs::write(&tmp, body) {
tracing::warn!(target: "openpxe::auth", "write api_key.json tmp: {e}");
return;
}
if let Err(e) = std::fs::rename(&tmp, self.path.as_path()) {
tracing::warn!(target: "openpxe::auth", "rename api_key.json: {e}");
}
}
}
/// 128 random bits as 32 lowercase hex chars — unambiguous to copy-paste
/// into an `x-api-key` header. UUID v4 is already our CSPRNG-backed source
/// for session ids, so no new dependency.
fn generate_key() -> String {
Uuid::new_v4().simple().to_string()
}
/// Length-checked constant-time byte compare — keeps key verification from
/// leaking the matched-prefix length via timing. A 128-bit random secret
/// isn't practically timing-attackable over a network, but the check is
/// four lines, so we keep it.
fn ct_eq(a: &[u8], b: &[u8]) -> bool {
if a.len() != b.len() {
return false;
}
let mut diff = 0u8;
for (x, y) in a.iter().zip(b.iter()) {
diff |= x ^ y;
}
diff == 0
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::tempdir;
#[test]
fn generates_persists_and_reloads() {
let dir = tempdir().unwrap();
let s = ApiKeyStore::load_or_init(dir.path());
let k = s.current();
assert_eq!(k.len(), 32, "32 hex chars = 128 bits");
assert!(s.verify(&k));
assert!(!s.verify("wrong"));
assert!(!s.verify(""), "blank header must not authenticate");
// Reload from disk → same key (survives restart).
let s2 = ApiKeyStore::load_or_init(dir.path());
assert_eq!(s2.current(), k);
}
#[test]
fn regenerate_invalidates_old() {
let dir = tempdir().unwrap();
let s = ApiKeyStore::load_or_init(dir.path());
let old = s.current();
let new = s.regenerate();
assert_ne!(old, new);
assert!(s.verify(&new));
assert!(!s.verify(&old), "old key must stop working");
// Persisted: a reload sees the new key.
let s2 = ApiKeyStore::load_or_init(dir.path());
assert_eq!(s2.current(), new);
}
}
+87 -2
View File
@@ -41,7 +41,16 @@ pub struct BootRule {
pub arch: String, pub arch: String,
/// Boot entry id (a `BootEntry::id`) or reserved menu name /// Boot entry id (a `BootEntry::id`) or reserved menu name
/// (`_local`, `_queue`, …) to chain to when this rule matches. /// (`_local`, `_queue`, …) to chain to when this rule matches.
/// May be empty for a rule that only pins a driver mode.
#[serde(default)]
pub target: String, pub target: String,
/// v0.7.1: optional first-boot binary pin — `""` (auto: let the
/// escalation ladder decide), `"firmware"`, `"builtin"`, or
/// `"shim"`. Lets an operator declare "this rack is all Secure
/// Boot → serve the signed chain immediately", skipping the
/// learn-by-failing walk entirely for known fleets.
#[serde(default)]
pub driver_mode: String,
/// Rules can be parked without deleting them. /// Rules can be parked without deleting them.
#[serde(default = "default_true")] #[serde(default = "default_true")]
pub enabled: bool, pub enabled: bool,
@@ -111,6 +120,7 @@ impl BootRulesStore {
r.mac_prefix = normalize_mac(&r.mac_prefix); r.mac_prefix = normalize_mac(&r.mac_prefix);
r.arch = r.arch.trim().to_ascii_lowercase(); r.arch = r.arch.trim().to_ascii_lowercase();
r.target = r.target.trim().to_string(); r.target = r.target.trim().to_string();
r.driver_mode = r.driver_mode.trim().to_ascii_lowercase();
r.note = r.note.trim().to_string(); r.note = r.note.trim().to_string();
} }
cfg.webhook_url = cfg.webhook_url.trim().to_string(); cfg.webhook_url = cfg.webhook_url.trim().to_string();
@@ -134,10 +144,40 @@ impl BootRulesStore {
/// passed one along, `None` otherwise (older chains). /// passed one along, `None` otherwise (older chains).
#[must_use] #[must_use]
pub fn match_target(&self, mac: &str, arch: Option<&str>) -> Option<String> { pub fn match_target(&self, mac: &str, arch: Option<&str>) -> Option<String> {
self.first_match(mac, arch, |r| {
(!r.target.is_empty()).then(|| r.target.clone())
})
}
/// v0.7.1: first enabled rule that pins a driver mode for `(mac,
/// arch)`. Consulted by the DHCP proxy *before* the automatic
/// escalation ladder — an operator who knows a rack is all Secure
/// Boot pins it to `shim` and those machines never walk the ladder.
/// Unknown mode strings are ignored (forward compatibility).
#[must_use]
pub fn driver_mode_hint(&self, mac: &str, arch: Option<&str>) -> Option<crate::DriverMode> {
self.first_match(mac, arch, |r| match r.driver_mode.as_str() {
"firmware" => Some(crate::DriverMode::Firmware),
"builtin" => Some(crate::DriverMode::Builtin),
"shim" => Some(crate::DriverMode::Shim),
_ => None,
})
}
/// Shared rule-matching walk: returns the first `extract` result from
/// an enabled rule whose selectors match. Rules that match but yield
/// `None` from `extract` (e.g. no target set, or no driver mode set)
/// don't stop the walk — target rules and mode-pin rules coexist.
fn first_match<T>(
&self,
mac: &str,
arch: Option<&str>,
extract: impl Fn(&BootRule) -> Option<T>,
) -> Option<T> {
let mac = normalize_mac(mac); let mac = normalize_mac(mac);
let g = self.inner.read(); let g = self.inner.read();
for r in &g.rules { for r in &g.rules {
if !r.enabled || r.target.is_empty() { if !r.enabled {
continue; continue;
} }
if !r.mac_prefix.is_empty() && !mac.starts_with(r.mac_prefix.as_str()) { if !r.mac_prefix.is_empty() && !mac.starts_with(r.mac_prefix.as_str()) {
@@ -151,7 +191,9 @@ impl BootRulesStore {
_ => continue, _ => continue,
} }
} }
return Some(r.target.clone()); if let Some(v) = extract(r) {
return Some(v);
}
} }
None None
} }
@@ -189,11 +231,54 @@ mod tests {
mac_prefix: mac_prefix.into(), mac_prefix: mac_prefix.into(),
arch: arch.into(), arch: arch.into(),
target: target.into(), target: target.into(),
driver_mode: String::new(),
enabled: true, enabled: true,
note: String::new(), note: String::new(),
} }
} }
#[test]
fn driver_mode_hint_pins_known_modes_and_ignores_unknown() {
let dir = tempdir().unwrap();
let s = BootRulesStore::load_or_default(dir.path());
let mut sb_rack = rule("aa:bb:cc", "", "");
sb_rack.driver_mode = "SHIM".into(); // normalized on replace
let mut weird = rule("11:22:33", "", "");
weird.driver_mode = "quantum".into(); // unknown → ignored
s.replace(BootRulesConfig {
rules: vec![sb_rack, weird],
webhook_url: String::new(),
});
assert_eq!(
s.driver_mode_hint("aa:bb:cc:00:00:01", None),
Some(crate::DriverMode::Shim)
);
assert_eq!(s.driver_mode_hint("11:22:33:00:00:01", None), None);
assert_eq!(s.driver_mode_hint("99:99:99:00:00:01", None), None);
}
#[test]
fn mode_pin_rule_does_not_shadow_later_target_rule() {
// A mode-only rule and a target rule can both apply to the same
// client: the mode pin must not consume the target walk.
let dir = tempdir().unwrap();
let s = BootRulesStore::load_or_default(dir.path());
let mut pin = rule("aa:bb", "", "");
pin.driver_mode = "builtin".into();
s.replace(BootRulesConfig {
rules: vec![pin, rule("aa:bb", "", "rack-image")],
webhook_url: String::new(),
});
assert_eq!(
s.driver_mode_hint("aa:bb:00:00:00:01", None),
Some(crate::DriverMode::Builtin)
);
assert_eq!(
s.match_target("aa:bb:00:00:00:01", None).as_deref(),
Some("rack-image")
);
}
#[test] #[test]
fn empty_config_matches_nothing() { fn empty_config_matches_nothing() {
let dir = tempdir().unwrap(); let dir = tempdir().unwrap();
+2
View File
@@ -2,6 +2,7 @@
//! runtime settings, and the Queued Deployment queue. //! runtime settings, and the Queued Deployment queue.
#![forbid(unsafe_code)] #![forbid(unsafe_code)]
pub mod api_key;
pub mod arch; pub mod arch;
pub mod auth; pub mod auth;
pub mod boot_log; pub mod boot_log;
@@ -23,6 +24,7 @@ pub mod settings;
pub mod sso; pub mod sso;
pub mod wol; pub mod wol;
pub use api_key::ApiKeyStore;
pub use arch::{ClientArch, DriverMode, FirmwareClass}; pub use arch::{ClientArch, DriverMode, FirmwareClass};
pub use auth::{AdminAccount, AdminPublic, AdminStore}; pub use auth::{AdminAccount, AdminPublic, AdminStore};
pub use boot_log::{BootEvent, BootLog}; pub use boot_log::{BootEvent, BootLog};
+5 -2
View File
@@ -15,7 +15,10 @@ tokio.workspace = true
socket2.workspace = true socket2.workspace = true
dhcproto.workspace = true dhcproto.workspace = true
tracing.workspace = true tracing.workspace = true
thiserror.workspace = true
anyhow.workspace = true anyhow.workspace = true
bytes.workspace = true
parking_lot.workspace = true parking_lot.workspace = true
# v0.7.1: learned driver modes persist to <work_dir>/driver_modes.json.
serde_json.workspace = true
[dev-dependencies]
tempfile = "3.12"
+305 -76
View File
@@ -1,24 +1,43 @@
//! Automatic per-MAC NIC driver-mode escalation (v0.6.1). //! Automatic per-MAC boot-binary escalation (v0.6.1, extended v0.7.x).
//! //!
//! OpenPXE serves the firmware-net iPXE build (`snponly`/`undionly`) by //! OpenPXE serves the firmware-net iPXE build (`snponly`/`undionly`) by
//! default — it's the most reliable choice for chainloading because the //! default — it's the most reliable choice for chainloading because the
//! firmware just proved its network works by downloading the NBP. A minority //! firmware just proved its network works by downloading the NBP. Two
//! of NICs have a missing or buggy firmware UNDI/SNP stack; those clients //! classes of machine can't run it:
//! TFTP the binary fine, but then iPXE can't bring the link up, so the
//! tell-tale second DHCP DISCOVER carrying the `iPXE` user-class never arrives
//! and the machine eventually re-PXE-boots.
//! //!
//! We detect exactly that: a *fresh* firmware DISCOVER from a MAC whose //! * a minority of NICs have a missing or buggy firmware UNDI/SNP stack —
//! previous firmware attempt was never confirmed by an iPXE handoff means the //! they TFTP the binary fine but iPXE can't bring the link up;
//! firmware-net build failed → escalate that MAC to [`DriverMode::Builtin`] //! * Secure-Boot firmware downloads it fine but refuses to *execute* an
//! (iPXE's own NIC drivers). The decision is sticky — once a MAC settles on a //! unsigned image.
//! mode that completes the handoff, later boots go straight to it. There is no //!
//! operator toggle; it just works, and the default (firmware) path is //! Both look identical from here: the tell-tale second DHCP DISCOVER
//! unchanged so hardware that already boots never regresses. //! carrying the `iPXE` user-class never arrives and the machine
//! re-PXE-boots. So a fresh firmware DISCOVER from a MAC whose previous
//! attempt was never confirmed climbs one rung:
//! `Firmware → Builtin → Shim` (the signed shim+GRUB chain). The decision
//! is sticky; there is no operator toggle; the default path is unchanged
//! so hardware that already boots never regresses.
//!
//! v0.7.1 — **learned modes persist**. Walking the ladder costs one or
//! two failed boot cycles, so a machine should pay it once *ever*, not
//! once per idle window or server restart. Two events pin a MAC's mode
//! to disk (`<work_dir>/driver_modes.json`):
//!
//! * a confirmed iPXE handoff at a non-default mode (Builtin proved to
//! work — also Shim, via the GRUB→iPXE same-boot chainload);
//! * reaching the terminal Shim rung (Secure-Boot machines never produce
//! an iPXE handoff from the signed menu, so escalation itself is the
//! best knowledge we'll ever have).
//!
//! Pinned entries are immune to the TTL and reload at startup. The
//! operator escape hatch is a rules-level driver-mode pin (which
//! overrides this table entirely) or deleting `driver_modes.json`.
use openpxe_core::DriverMode; use openpxe_core::DriverMode;
use parking_lot::Mutex; use parking_lot::Mutex;
use std::collections::HashMap; use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
/// Multiple DISCOVERs within this window belong to the *same* boot (DHCP /// Multiple DISCOVERs within this window belong to the *same* boot (DHCP
@@ -26,13 +45,14 @@ use std::time::{Duration, Instant};
/// DISCOVER). They must not be mistaken for a failed-and-retried boot. /// DISCOVER). They must not be mistaken for a failed-and-retried boot.
const SAME_BOOT_DEBOUNCE: Duration = Duration::from_secs(8); const SAME_BOOT_DEBOUNCE: Duration = Duration::from_secs(8);
/// Forget a MAC's state after this long with no activity, so a transient /// Forget an *unpinned* MAC's state after this long with no activity, so
/// escalation doesn't pin a client to Builtin forever and the map stays /// a transient mid-walk state doesn't linger and the map stays bounded.
/// bounded over a long-running deployment. /// Pinned (learned) entries are exempt — that's their whole point.
const ENTRY_TTL: Duration = Duration::from_mins(30); const ENTRY_TTL: Duration = Duration::from_mins(30);
/// Hard cap on tracked MACs. Past this we evict the least-recently-seen /// Hard cap on tracked MACs. Past this we evict the least-recently-seen
/// entry — escalation is best-effort, never a memory-growth vector. /// entry (unpinned first) — escalation is best-effort, never a
/// memory-growth vector.
const MAX_ENTRIES: usize = 4096; const MAX_ENTRIES: usize = 4096;
/// How often (at most) the whole map is swept for expired entries. /// How often (at most) the whole map is swept for expired entries.
@@ -49,6 +69,8 @@ struct Entry {
/// confirm it worked. A *new* boot arriving while this is still true means /// confirm it worked. A *new* boot arriving while this is still true means
/// the previous attempt failed and we should escalate. /// the previous attempt failed and we should escalate.
awaiting_confirm: bool, awaiting_confirm: bool,
/// Learned mode (v0.7.1): persisted to disk, exempt from the TTL.
pinned: bool,
last_seen: Instant, last_seen: Instant,
} }
@@ -72,14 +94,68 @@ impl Default for Inner {
#[derive(Debug, Default)] #[derive(Debug, Default)]
pub struct DriverEscalation { pub struct DriverEscalation {
inner: Mutex<Inner>, inner: Mutex<Inner>,
/// Persistence target for learned modes; `None` = ephemeral (tests).
path: Option<Arc<PathBuf>>,
} }
impl DriverEscalation { impl DriverEscalation {
/// Ephemeral instance (no persistence) — used by tests.
#[must_use] #[must_use]
pub fn new() -> Self { pub fn new() -> Self {
Self::default() Self::default()
} }
/// Instance backed by `<work_dir>/driver_modes.json`. Learned modes
/// from previous runs are reloaded as pinned entries; a missing or
/// corrupt file starts empty (same crash-cache policy as every other
/// store — a bad file must never block PXE).
#[must_use]
pub fn load_or_default(work_dir: &Path) -> Self {
let path = work_dir.join("driver_modes.json");
let mut map = HashMap::new();
if let Ok(text) = std::fs::read_to_string(&path) {
match serde_json::from_str::<HashMap<String, DriverMode>>(&text) {
Ok(loaded) => {
let now = Instant::now();
for (mac, mode) in loaded {
// Firmware is the default — persisting it would be
// noise; tolerate it in the file but don't track it.
if mode == DriverMode::Firmware {
continue;
}
map.insert(
mac,
Entry {
mode,
awaiting_confirm: false,
pinned: true,
last_seen: now,
},
);
}
tracing::info!(
target: "openpxe::dhcp",
learned = map.len(),
"loaded learned driver modes"
);
}
Err(e) => {
tracing::warn!(
target: "openpxe::dhcp",
"driver_modes.json present but unreadable ({e}); starting empty"
);
}
}
}
Self {
inner: Mutex::new(Inner {
map,
last_prune: Instant::now(),
}),
path: Some(Arc::new(path)),
}
}
/// Decide the driver mode for a firmware (PXEClient/HTTPClient) boot from /// Decide the driver mode for a firmware (PXEClient/HTTPClient) boot from
/// `mac`. `primary` is true for the main DHCP DISCOVER (:67) and false for /// `mac`. `primary` is true for the main DHCP DISCOVER (:67) and false for
/// the PXE Boot Server query (:4011); only the primary path drives /// the PXE Boot Server query (:4011); only the primary path drives
@@ -91,82 +167,150 @@ impl DriverEscalation {
/// Record that `mac` completed the iPXE handoff (a DISCOVER carrying the /// Record that `mac` completed the iPXE handoff (a DISCOVER carrying the
/// `iPXE` user-class). The mode we last served worked, so stop awaiting /// `iPXE` user-class). The mode we last served worked, so stop awaiting
/// confirmation and keep it sticky for next time. /// confirmation, keep it sticky, and — for non-default modes — pin it to
/// disk so the machine never re-walks the ladder (v0.7.1).
pub fn mark_ipxe_success(&self, mac: &str) { pub fn mark_ipxe_success(&self, mac: &str) {
self.confirm_at(mac, Instant::now()); self.confirm_at(mac, Instant::now());
} }
fn decide_at(&self, mac: &str, primary: bool, now: Instant) -> DriverMode { fn decide_at(&self, mac: &str, primary: bool, now: Instant) -> DriverMode {
let mut g = self.inner.lock(); let (mode, snapshot) = {
if now.duration_since(g.last_prune) >= PRUNE_INTERVAL { let mut g = self.inner.lock();
g.map if now.duration_since(g.last_prune) >= PRUNE_INTERVAL {
.retain(|_, e| now.duration_since(e.last_seen) < ENTRY_TTL); g.map
g.last_prune = now; .retain(|_, e| e.pinned || now.duration_since(e.last_seen) < ENTRY_TTL);
} g.last_prune = now;
// Inline staleness check: a MAC whose entry outlived the TTL starts }
// fresh even when the amortized sweep above hasn't caught it yet. // Inline staleness check: an unpinned MAC whose entry outlived
if g.map // the TTL starts fresh even when the amortized sweep above
.get(mac) // hasn't caught it yet. Pinned entries never go stale.
.is_some_and(|e| now.duration_since(e.last_seen) >= ENTRY_TTL) if g.map
{ .get(mac)
g.map.remove(mac); .is_some_and(|e| !e.pinned && now.duration_since(e.last_seen) >= ENTRY_TTL)
} {
g.map.remove(mac);
}
match g.map.get_mut(mac) { let mut newly_pinned = false;
None => { let mode = match g.map.get_mut(mac) {
g.map.insert( None => {
mac.to_owned(), g.map.insert(
Entry { mac.to_owned(),
mode: DriverMode::Firmware, Entry {
// Only the primary DISCOVER opens a confirmation window. mode: DriverMode::Firmware,
awaiting_confirm: primary, // Only the primary DISCOVER opens a confirmation window.
last_seen: now, awaiting_confirm: primary,
}, pinned: false,
); last_seen: now,
if g.map.len() > MAX_ENTRIES { },
evict_oldest(&mut g.map); );
} if g.map.len() > MAX_ENTRIES {
DriverMode::Firmware evict_oldest(&mut g.map);
}
Some(entry) => {
let recent = now.duration_since(entry.last_seen) < SAME_BOOT_DEBOUNCE;
if primary && !recent {
// A genuinely new boot. If the previous attempt was never
// confirmed, the build we served failed → climb one rung:
// Firmware (firmware NIC stack) → Builtin (iPXE's own
// drivers) → Shim (signed shim+GRUB, v0.7.0 — covers
// Secure Boot firmware that downloads our unsigned iPXE
// but refuses to execute it). Shim is terminal: a MAC
// there stays until its entry TTLs out and resets.
if entry.awaiting_confirm {
entry.mode = match entry.mode {
DriverMode::Firmware => DriverMode::Builtin,
DriverMode::Builtin | DriverMode::Shim => DriverMode::Shim,
};
} }
entry.awaiting_confirm = true; DriverMode::Firmware
} }
entry.last_seen = now; Some(entry) => {
entry.mode let recent = now.duration_since(entry.last_seen) < SAME_BOOT_DEBOUNCE;
} if primary && !recent {
// A genuinely new boot. If the previous attempt was
// never confirmed, the build we served failed → climb
// one rung: Firmware (firmware NIC stack) → Builtin
// (iPXE's own drivers) → Shim (signed shim+GRUB —
// covers Secure Boot firmware that downloads our
// unsigned iPXE but refuses to execute it). Shim is
// terminal and pins to disk: SB machines never emit
// an iPXE handoff from the signed menu, so reaching
// the rung *is* the durable knowledge.
if entry.awaiting_confirm {
entry.mode = match entry.mode {
DriverMode::Firmware => DriverMode::Builtin,
DriverMode::Builtin | DriverMode::Shim => DriverMode::Shim,
};
if entry.mode == DriverMode::Shim && !entry.pinned {
entry.pinned = true;
newly_pinned = true;
}
}
entry.awaiting_confirm = true;
}
entry.last_seen = now;
entry.mode
}
};
(mode, newly_pinned.then(|| pinned_snapshot(&g.map)))
};
if let Some(s) = snapshot {
self.persist(&s);
} }
mode
} }
fn confirm_at(&self, mac: &str, now: Instant) { fn confirm_at(&self, mac: &str, now: Instant) {
let mut g = self.inner.lock(); let snapshot = {
if let Some(e) = g.map.get_mut(mac) { let mut g = self.inner.lock();
let Some(e) = g.map.get_mut(mac) else {
return;
};
e.awaiting_confirm = false; e.awaiting_confirm = false;
e.last_seen = now; e.last_seen = now;
// A proven non-default mode is worth remembering forever —
// the machine demonstrably can't use the default path.
if e.mode != DriverMode::Firmware && !e.pinned {
e.pinned = true;
Some(pinned_snapshot(&g.map))
} else {
None
}
};
if let Some(s) = snapshot {
self.persist(&s);
}
}
/// Best-effort atomic write of the learned-mode table. No-op for
/// ephemeral instances. Failure logs and moves on — persistence is an
/// optimization, never a correctness requirement.
fn persist(&self, snapshot: &HashMap<String, DriverMode>) {
let Some(path) = &self.path else { return };
let body = match serde_json::to_vec_pretty(snapshot) {
Ok(b) => b,
Err(e) => {
tracing::warn!(target: "openpxe::dhcp", "serialize driver_modes.json: {e}");
return;
}
};
if let Some(parent) = path.parent() {
let _ = std::fs::create_dir_all(parent);
}
let tmp = path.with_extension("json.tmp");
if let Err(e) = std::fs::write(&tmp, body) {
tracing::warn!(target: "openpxe::dhcp", "write driver_modes.json tmp: {e}");
return;
}
if let Err(e) = std::fs::rename(&tmp, path.as_path()) {
tracing::warn!(target: "openpxe::dhcp", "rename driver_modes.json: {e}");
} }
} }
} }
fn pinned_snapshot(map: &HashMap<String, Entry>) -> HashMap<String, DriverMode> {
map.iter()
.filter(|(_, e)| e.pinned)
.map(|(k, e)| (k.clone(), e.mode))
.collect()
}
fn evict_oldest(map: &mut HashMap<String, Entry>) { fn evict_oldest(map: &mut HashMap<String, Entry>) {
if let Some(oldest) = map // Prefer evicting an unpinned entry; only touch learned modes when
.iter() // the whole table is pinned (4096 learned machines — at that point
.min_by_key(|(_, e)| e.last_seen) // the operator has bigger questions than our memory bound).
.map(|(k, _)| k.clone()) let pick = |pinned: bool| {
{ map.iter()
.filter(|(_, e)| e.pinned == pinned)
.min_by_key(|(_, e)| e.last_seen)
.map(|(k, _)| k.clone())
};
if let Some(oldest) = pick(false).or_else(|| pick(true)) {
map.remove(&oldest); map.remove(&oldest);
} }
} }
@@ -174,6 +318,7 @@ fn evict_oldest(map: &mut HashMap<String, Entry>) {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use tempfile::tempdir;
#[test] #[test]
fn firmware_first_then_escalates_on_unconfirmed_retry() { fn firmware_first_then_escalates_on_unconfirmed_retry() {
@@ -250,7 +395,7 @@ mod tests {
} }
#[test] #[test]
fn stale_entry_is_forgotten_and_resets_to_firmware() { fn stale_unpinned_entry_is_forgotten_and_resets_to_firmware() {
let e = DriverEscalation::new(); let e = DriverEscalation::new();
let t0 = Instant::now(); let t0 = Instant::now();
assert_eq!(e.decide_at("dd", true, t0), DriverMode::Firmware); assert_eq!(e.decide_at("dd", true, t0), DriverMode::Firmware);
@@ -258,8 +403,92 @@ mod tests {
e.decide_at("dd", true, t0 + Duration::from_mins(1)), e.decide_at("dd", true, t0 + Duration::from_mins(1)),
DriverMode::Builtin DriverMode::Builtin
); );
// After the TTL with no activity the entry is pruned → fresh firmware. // After the TTL with no activity the (unpinned) Builtin walk is
// pruned → fresh firmware. (A *confirmed* Builtin would be pinned
// and survive — see learned_builtin_survives_ttl.)
let later = t0 + Duration::from_mins(1) + ENTRY_TTL + Duration::from_secs(1); let later = t0 + Duration::from_mins(1) + ENTRY_TTL + Duration::from_secs(1);
assert_eq!(e.decide_at("dd", true, later), DriverMode::Firmware); assert_eq!(e.decide_at("dd", true, later), DriverMode::Firmware);
} }
#[test]
fn shim_pin_survives_ttl() {
// v0.7.1: reaching the Shim rung is durable knowledge — the
// machine must NOT re-walk the ladder after an idle period.
let e = DriverEscalation::new();
let t0 = Instant::now();
let _ = e.decide_at("ff", true, t0);
let _ = e.decide_at("ff", true, t0 + Duration::from_mins(1));
assert_eq!(
e.decide_at("ff", true, t0 + Duration::from_mins(2)),
DriverMode::Shim
);
let much_later = t0 + Duration::from_mins(2) + ENTRY_TTL + Duration::from_mins(5);
assert_eq!(e.decide_at("ff", true, much_later), DriverMode::Shim);
}
#[test]
fn learned_builtin_survives_ttl() {
let e = DriverEscalation::new();
let t0 = Instant::now();
let _ = e.decide_at("gg", true, t0);
assert_eq!(
e.decide_at("gg", true, t0 + Duration::from_mins(1)),
DriverMode::Builtin
);
// The handoff confirms Builtin → pinned.
e.confirm_at("gg", t0 + Duration::from_secs(61));
let much_later = t0 + ENTRY_TTL + Duration::from_mins(10);
assert_eq!(e.decide_at("gg", true, much_later), DriverMode::Builtin);
}
#[test]
fn learned_modes_persist_across_restart() {
let dir = tempdir().unwrap();
let t0 = Instant::now();
{
let e = DriverEscalation::load_or_default(dir.path());
// Walk one MAC to Shim (pins on escalation)...
let _ = e.decide_at("aa:01", true, t0);
let _ = e.decide_at("aa:01", true, t0 + Duration::from_mins(1));
assert_eq!(
e.decide_at("aa:01", true, t0 + Duration::from_mins(2)),
DriverMode::Shim
);
// ...and another to a confirmed Builtin (pins on handoff).
let _ = e.decide_at("aa:02", true, t0);
let _ = e.decide_at("aa:02", true, t0 + Duration::from_mins(1));
e.confirm_at("aa:02", t0 + Duration::from_secs(61));
}
// "Restart": a fresh instance from the same work_dir knows both.
let e2 = DriverEscalation::load_or_default(dir.path());
assert_eq!(e2.decide_at("aa:01", true, t0), DriverMode::Shim);
assert_eq!(e2.decide_at("aa:02", true, t0), DriverMode::Builtin);
// Unlearned MACs still start at the default.
assert_eq!(e2.decide_at("aa:03", true, t0), DriverMode::Firmware);
}
#[test]
fn corrupt_persistence_file_starts_empty() {
let dir = tempdir().unwrap();
std::fs::write(dir.path().join("driver_modes.json"), b"{broken").unwrap();
let e = DriverEscalation::load_or_default(dir.path());
assert_eq!(
e.decide_at("aa:bb", true, Instant::now()),
DriverMode::Firmware
);
}
#[test]
fn confirmed_firmware_is_not_persisted() {
// The default mode is never written — the file only carries
// exceptions, so a healthy fleet leaves it absent/empty.
let dir = tempdir().unwrap();
let t0 = Instant::now();
{
let e = DriverEscalation::load_or_default(dir.path());
let _ = e.decide_at("aa:09", true, t0);
e.confirm_at("aa:09", t0 + Duration::from_secs(2));
}
assert!(!dir.path().join("driver_modes.json").exists());
}
} }
+27 -7
View File
@@ -5,7 +5,9 @@ use crate::escalation::DriverEscalation;
use crate::reply::{build_reply, decide, BootDirective, ReplyContext}; use crate::reply::{build_reply, decide, BootDirective, ReplyContext};
use dhcproto::v4::{DhcpOption, Message, OptionCode}; use dhcproto::v4::{DhcpOption, Message, OptionCode};
use dhcproto::{Decodable, Decoder, Encodable, Encoder}; use dhcproto::{Decodable, Decoder, Encodable, Encoder};
use openpxe_core::{ClientArch, ClientEvent, ClientRegistry, DriverMode, FirmwareClass}; use openpxe_core::{
BootRulesStore, ClientArch, ClientEvent, ClientRegistry, DriverMode, FirmwareClass,
};
use socket2::{Domain, Protocol, Socket, Type}; use socket2::{Domain, Protocol, Socket, Type};
use std::net::{IpAddr, Ipv4Addr, SocketAddr, SocketAddrV4}; use std::net::{IpAddr, Ipv4Addr, SocketAddr, SocketAddrV4};
use std::sync::Arc; use std::sync::Arc;
@@ -19,12 +21,19 @@ pub struct DhcpProxyServer {
public_base_url: String, public_base_url: String,
clients: Arc<ClientRegistry>, clients: Arc<ClientRegistry>,
metrics: openpxe_core::Metrics, metrics: openpxe_core::Metrics,
/// Automatic per-MAC NIC driver-mode escalation (v0.6.1). Shared across /// Automatic per-MAC NIC driver-mode escalation (v0.6.1; persistent
/// the :67 and :4011 listener tasks via the server `Arc`. /// learned modes since v0.7.1). Shared across the :67 and :4011
/// listener tasks via the server `Arc`. Built by the caller so the
/// persistence path comes from the configured work dir.
escalation: DriverEscalation, escalation: DriverEscalation,
/// v0.7.1: boot rules — consulted for an operator driver-mode pin
/// (e.g. "this OUI is all Secure Boot → serve shim immediately")
/// before the automatic escalation ladder.
rules: BootRulesStore,
} }
impl DhcpProxyServer { impl DhcpProxyServer {
#[allow(clippy::too_many_arguments)]
pub fn new( pub fn new(
bind: IpAddr, bind: IpAddr,
dhcp_port: u16, dhcp_port: u16,
@@ -33,6 +42,8 @@ impl DhcpProxyServer {
public_base_url: String, public_base_url: String,
clients: Arc<ClientRegistry>, clients: Arc<ClientRegistry>,
metrics: openpxe_core::Metrics, metrics: openpxe_core::Metrics,
escalation: DriverEscalation,
rules: BootRulesStore,
) -> Self { ) -> Self {
Self { Self {
bind, bind,
@@ -42,7 +53,8 @@ impl DhcpProxyServer {
public_base_url, public_base_url,
clients, clients,
metrics, metrics,
escalation: DriverEscalation::new(), escalation,
rules,
} }
} }
@@ -142,9 +154,17 @@ impl DhcpProxyServer {
self.escalation.mark_ipxe_success(&mac); self.escalation.mark_ipxe_success(&mac);
DriverMode::Firmware // unused: this path serves the HTTP script DriverMode::Firmware // unused: this path serves the HTTP script
} }
FirmwareClass::PxeClient | FirmwareClass::HttpClient => self FirmwareClass::PxeClient | FirmwareClass::HttpClient => {
.escalation // v0.7.1: an operator rule pin wins over (and bypasses)
.mode_for_firmware_attempt(&mac, label == "67"), // the automatic escalation ladder — known Secure-Boot
// fleets boot the signed chain on the very first cycle.
if let Some(pinned) = self.rules.driver_mode_hint(&mac, Some(arch.as_str())) {
pinned
} else {
self.escalation
.mode_for_firmware_attempt(&mac, label == "67")
}
}
// Unreachable: FirmwareClass::Other returned above. // Unreachable: FirmwareClass::Other returned above.
FirmwareClass::Other => DriverMode::Firmware, FirmwareClass::Other => DriverMode::Firmware,
}; };
+4 -2
View File
@@ -25,11 +25,9 @@ time.workspace = true
axum.workspace = true axum.workspace = true
tower.workspace = true tower.workspace = true
tower-http.workspace = true tower-http.workspace = true
hyper.workspace = true
serde.workspace = true serde.workspace = true
serde_json.workspace = true serde_json.workspace = true
tracing.workspace = true tracing.workspace = true
thiserror.workspace = true
anyhow.workspace = true anyhow.workspace = true
bytes.workspace = true bytes.workspace = true
futures.workspace = true futures.workspace = true
@@ -49,6 +47,10 @@ base64.workspace = true
tokio = { workspace = true, features = ["macros", "rt", "rt-multi-thread", "time"] } tokio = { workspace = true, features = ["macros", "rt", "rt-multi-thread", "time"] }
tower = { workspace = true } tower = { workspace = true }
tempfile = "3.12" tempfile = "3.12"
# v0.7.4: probe-based introspection verifies kernel paths against the
# real ISO9660 tree, so the full-flow tests synthesize images with the
# shared test builder instead of label-only blobs.
openpxe-iso-store = { workspace = true, features = ["test-image"] }
serde_json = { workspace = true } serde_json = { workspace = true }
time = { workspace = true } time = { workspace = true }
# v0.4.61: integration tests need to generate real PNG bytes for the # v0.4.61: integration tests need to generate real PNG bytes for the
+107 -17
View File
@@ -19,7 +19,6 @@ use crate::ipxe_script::{
render_entry, render_family_menu, render_local_hdd, render_menu, render_nic_info, render_entry, render_family_menu, render_local_hdd, render_menu, render_nic_info,
render_queue_entry, render_shell, render_tools_menu, render_util, render_queue_entry, render_shell, render_tools_menu, render_util,
}; };
use crate::iso_fs;
use crate::log_stream; use crate::log_stream;
use crate::state::AppState; use crate::state::AppState;
use crate::terminal; use crate::terminal;
@@ -36,6 +35,7 @@ use openpxe_core::{
LogoSlot, NotifyConfig, Settings, SsoConfig, ALLOWED_LOGO_MIMES, MAX_LOGO_BYTES, LogoSlot, NotifyConfig, Settings, SsoConfig, ALLOWED_LOGO_MIMES, MAX_LOGO_BYTES,
}; };
use openpxe_ipxe_assets::asset_slice; use openpxe_ipxe_assets::asset_slice;
use openpxe_iso_store::iso_fs;
use openpxe_iso_store::{ use openpxe_iso_store::{
render_template, IsoCategory, IsoMeta, IsoSource, NfsAddRequest, SftpAddRequest, SmbAddRequest, render_template, IsoCategory, IsoMeta, IsoSource, NfsAddRequest, SftpAddRequest, SmbAddRequest,
SmbState, UnattendedKind, UnattendedMeta, SmbState, UnattendedKind, UnattendedMeta,
@@ -147,6 +147,14 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/logout", post(auth_api::api_logout)) .route("/api/logout", post(auth_api::api_logout))
.route("/api/me", get(auth_api::api_me)) .route("/api/me", get(auth_api::api_me))
.route("/api/me/credentials", put(auth_api::api_update_credentials)) .route("/api/me/credentials", put(auth_api::api_update_credentials))
// v0.8.0: operator API key surface (read current + regenerate).
// Gated by require_auth like the rest of /api/*; a logged-in
// operator or an x-api-key holder can read/rotate it.
.route("/api/api-key", get(auth_api::api_api_key_get))
.route(
"/api/api-key/regenerate",
post(auth_api::api_api_key_regenerate),
)
// SAML SSO configuration (FleetDM-shaped). Gated behind auth — the // SAML SSO configuration (FleetDM-shaped). Gated behind auth — the
// operator pastes their IdP metadata, Entity ID, and toggles here. // operator pastes their IdP metadata, Entity ID, and toggles here.
.route("/api/sso", get(api_sso_get).put(api_sso_put)) .route("/api/sso", get(api_sso_get).put(api_sso_put))
@@ -1087,28 +1095,105 @@ async fn iso_file(
State(state): State<AppState>, State(state): State<AppState>,
AxumPath((id, path)): AxumPath<(String, String)>, AxumPath((id, path)): AxumPath<(String, String)>,
) -> Response { ) -> Response {
// In-ISO file extraction is only supported for local ISOs — it let Some(meta) = state.iso_store.get(&id) else {
// needs random-access reads into the ISO9660 directory tree, which
// smbclient's whole-file streaming can't do efficiently. SMB-
// sourced ISOs use the raw streaming endpoint above instead.
let Some(iso_path) = state.iso_store.iso_path_for(&id) else {
return (StatusCode::NOT_FOUND, "no such iso").into_response(); return (StatusCode::NOT_FOUND, "no such iso").into_response();
}; };
let p = iso_path.clone();
let in_path = format!("/{path}"); let in_path = format!("/{path}");
let loc = tokio::task::spawn_blocking(move || iso_fs::lookup(&p, &in_path)) match &meta.source {
.await IsoSource::Local => {
.ok() let Some(iso_path) = state.iso_store.local_path(&meta) else {
.flatten(); return (StatusCode::NOT_FOUND, "no such iso").into_response();
let Some(loc) = loc else { };
return (StatusCode::NOT_FOUND, "not found inside iso").into_response(); let p = iso_path.clone();
}; let loc = tokio::task::spawn_blocking(move || iso_fs::lookup_local(&p, &in_path))
match stream_byte_range(&iso_path, loc.offset, loc.length).await { .await
Ok(r) => r, .ok()
Err(e) => (StatusCode::INTERNAL_SERVER_ERROR, format!("{e}")).into_response(), .flatten();
let Some(loc) = loc else {
return (StatusCode::NOT_FOUND, "not found inside iso").into_response();
};
match stream_byte_range(&iso_path, loc.offset, loc.length).await {
Ok(r) => r,
Err(e) => (StatusCode::INTERNAL_SERVER_ERROR, format!("{e}")).into_response(),
}
}
// v0.7.4: remote ISOs serve in-ISO files too — the same ISO9660
// walk runs over NFS READ3 / SFTP seek-reads, then the located
// byte range streams through the share manager. This is what
// makes the verified kernel/initrd boot entries on share-hosted
// Linux ISOs actually bootable.
IsoSource::Nfs {
share_id,
relative_path,
} => {
match state
.nfs_shares
.locate_in_iso(share_id, relative_path, &in_path)
.await
{
Ok(Some(loc)) => {
match state
.nfs_shares
.stream_iso(share_id, relative_path, loc.offset, Some(loc.length))
.await
{
Ok(stream) => in_iso_stream_response(Body::from_stream(stream), loc.length),
Err(e) => {
(StatusCode::BAD_GATEWAY, format!("nfs stream: {e}")).into_response()
}
}
}
Ok(None) => (StatusCode::NOT_FOUND, "not found inside iso").into_response(),
Err(e) => (StatusCode::BAD_GATEWAY, format!("nfs lookup: {e}")).into_response(),
}
}
IsoSource::Sftp {
share_id,
relative_path,
} => {
match state
.sftp_shares
.locate_in_iso(share_id, relative_path, &in_path)
.await
{
Ok(Some(loc)) => {
match state
.sftp_shares
.stream_iso(share_id, relative_path, loc.offset, Some(loc.length))
.await
{
Ok(stream) => in_iso_stream_response(Body::from_stream(stream), loc.length),
Err(e) => {
(StatusCode::BAD_GATEWAY, format!("sftp stream: {e}")).into_response()
}
}
}
Ok(None) => (StatusCode::NOT_FOUND, "not found inside iso").into_response(),
Err(e) => (StatusCode::BAD_GATEWAY, format!("sftp lookup: {e}")).into_response(),
}
}
// smbclient streams sequentially — no seeks, no ISO9660 walk.
// SMB ISOs never emit kernel entries, so nothing requests this.
IsoSource::Smb { .. } => (
StatusCode::NOT_FOUND,
"in-ISO files are not available for SMB-sourced ISOs",
)
.into_response(),
} }
} }
/// 200 response wrapping an in-ISO byte-range stream from a share
/// manager. Content-Length is the located file's length — the stream is
/// already bounded to exactly that range.
fn in_iso_stream_response(body: Body, length: u64) -> Response {
Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, "application/octet-stream")
.header(header::CONTENT_LENGTH, length)
.body(body)
.unwrap()
}
async fn stream_file_range( async fn stream_file_range(
path: &std::path::Path, path: &std::path::Path,
range: Option<&HeaderValue>, range: Option<&HeaderValue>,
@@ -1867,6 +1952,10 @@ async fn api_docs() -> Json<serde_json::Value> {
"summary": "Auth status — { setup_required, authenticated, user }. Always 200."}, "summary": "Auth status — { setup_required, authenticated, user }. Always 200."},
{"method": "PUT", "path": "/api/me/credentials", {"method": "PUT", "path": "/api/me/credentials",
"summary": "Rotate the admin's credentials. Body: { current_password, new_username?, new_password? }. Revokes all other sessions on success."}, "summary": "Rotate the admin's credentials. Body: { current_password, new_username?, new_password? }. Revokes all other sessions on success."},
{"method": "GET", "path": "/api/api-key",
"summary": "Return the operator API key + the header to send it in (x-api-key). That header authenticates API calls without a browser session — full operator access."},
{"method": "POST", "path": "/api/api-key/regenerate",
"summary": "Mint a fresh API key, invalidating the previous one immediately."},
], ],
}, },
{ {
@@ -2744,6 +2833,7 @@ async fn api_network(State(state): State<AppState>) -> Json<serde_json::Value> {
.strip_prefix("http://") .strip_prefix("http://")
.unwrap_or(&state.public_base_url), .unwrap_or(&state.public_base_url),
"nic_name": state.nic_name, "nic_name": state.nic_name,
"nic_link": state.nic_link,
"subnet_mask": state.subnet_mask, "subnet_mask": state.subnet_mask,
"gateway": state.gateway, "gateway": state.gateway,
"dns_server": state.settings.snapshot().dns_server, "dns_server": state.settings.snapshot().dns_server,
+40 -7
View File
@@ -52,6 +52,12 @@ const SESSION_TTL: Duration = Duration::from_hours(24);
/// to avoid collisions with anything else sharing the host. /// to avoid collisions with anything else sharing the host.
pub const SESSION_COOKIE: &str = "openpxe_session"; pub const SESSION_COOKIE: &str = "openpxe_session";
/// Header an API client sends to authenticate without a browser session.
/// Matches the de-facto `x-api-key` convention operators already use with
/// other appliances. A valid key grants the same access as a logged-in
/// operator. See [`crate::state::AppState::api_key`].
pub const API_KEY_HEADER: &str = "x-api-key";
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
struct Session { struct Session {
username: String, username: String,
@@ -223,13 +229,18 @@ pub async fn require_auth(
if is_public_path(path) { if is_public_path(path) {
return next.run(req).await; return next.run(req).await;
} }
// Authenticated path. The cookie must be present, map to a live // Authenticated path: either a live operator session cookie (the
// session, and the TTL refresh happens as a side-effect. // browser) or the x-api-key header (scripts / Postman). Touching the
let token = parse_cookie(req.headers()); // cookie refreshes its idle TTL as a side-effect.
if let Some(t) = token { let session_ok =
if state.sessions.touch(&t).is_some() { parse_cookie(req.headers()).is_some_and(|t| state.sessions.touch(&t).is_some());
return next.run(req).await; let key_ok = req
} .headers()
.get(API_KEY_HEADER)
.and_then(|v| v.to_str().ok())
.is_some_and(|k| state.api_key.verify(k));
if session_ok || key_ok {
return next.run(req).await;
} }
( (
StatusCode::UNAUTHORIZED, StatusCode::UNAUTHORIZED,
@@ -447,6 +458,28 @@ pub async fn api_update_credentials(
} }
} }
/// Return the current operator API key plus the header to send it in.
/// Gated by the auth middleware, so only a logged-in operator (or a
/// caller already holding the key) can read it.
pub async fn api_api_key_get(State(state): State<AppState>) -> Response {
(
StatusCode::OK,
Json(json!({ "key": state.api_key.current(), "header": API_KEY_HEADER })),
)
.into_response()
}
/// Mint a fresh operator API key, invalidating the previous one, and
/// return it. Same gating as the GET.
pub async fn api_api_key_regenerate(State(state): State<AppState>) -> Response {
let key = state.api_key.regenerate();
(
StatusCode::OK,
Json(json!({ "key": key, "header": API_KEY_HEADER })),
)
.into_response()
}
#[derive(Debug, Serialize)] #[derive(Debug, Serialize)]
struct LoginPayload<'a> { struct LoginPayload<'a> {
user: &'a AdminPublic, user: &'a AdminPublic,
+36
View File
@@ -35,6 +35,24 @@ pub fn render_grub_menu(isos: &[IsoMeta], base_url: &str) -> String {
let dev = grub_http_device(base); let dev = grub_http_device(base);
let mut s = String::new(); let mut s = String::new();
let _ = writeln!(s, "# OpenPXE — Secure Boot menu (signed shim+GRUB chain)"); let _ = writeln!(s, "# OpenPXE — Secure Boot menu (signed shim+GRUB chain)");
// v0.7.1: before showing the limited signed menu, try to hand the
// boot back to full iPXE *in this same boot cycle*. With Secure Boot
// OFF the chainload succeeds and the client gets the complete iPXE
// feature set (sanboot, wimboot, the full menu) despite having been
// escalated here. With Secure Boot ON, shim's verifier refuses the
// unsigned image INLINE — no reboot, no failed cycle — and execution
// falls through to the signed menu below. The all-drivers build is
// used because a MAC only lands here after the firmware-net build
// already failed once.
let _ = writeln!(s, "if [ \"$grub_cpu\" = \"arm64\" ]; then");
let _ = writeln!(s, " set openpxe_ipxe=ipxe-arm64.efi");
let _ = writeln!(s, "else");
let _ = writeln!(s, " set openpxe_ipxe=ipxe.efi");
let _ = writeln!(s, "fi");
let _ = writeln!(s, "if chainloader {dev}/ipxe/$openpxe_ipxe ; then");
let _ = writeln!(s, " boot");
let _ = writeln!(s, "fi");
let _ = writeln!(s);
let _ = writeln!(s, "set timeout=30"); let _ = writeln!(s, "set timeout=30");
let _ = writeln!(s, "set default=0"); let _ = writeln!(s, "set default=0");
let _ = writeln!(s); let _ = writeln!(s);
@@ -139,6 +157,24 @@ mod tests {
assert!(cfg.contains("Boot from local disk"), "{cfg}"); assert!(cfg.contains("Boot from local disk"), "{cfg}");
} }
#[test]
fn config_tries_ipxe_chainload_before_menu() {
// v0.7.1: SB-off machines recover full iPXE in the same boot;
// SB-on machines fail the chainload inline and reach the menu.
let cfg = render_grub_menu(&[linux_iso()], "http://10.0.0.5:8080");
let chain_pos = cfg
.find("if chainloader (http,10.0.0.5:8080)/ipxe/$openpxe_ipxe ; then")
.expect("chainload attempt missing");
let menu_pos = cfg.find("menuentry").expect("menu missing");
assert!(
chain_pos < menu_pos,
"chainload must precede the menu:\n{cfg}"
);
// Arch-conditional binary selection via GRUB's $grub_cpu.
assert!(cfg.contains("set openpxe_ipxe=ipxe-arm64.efi"), "{cfg}");
assert!(cfg.contains("set openpxe_ipxe=ipxe.efi"), "{cfg}");
}
#[test] #[test]
fn sanboot_and_wimboot_entries_are_omitted() { fn sanboot_and_wimboot_entries_are_omitted() {
let mut iso = linux_iso(); let mut iso = linux_iso();
-135
View File
@@ -1,135 +0,0 @@
//! Minimal read-only ISO9660 lookup. Given an uploaded ISO file and an
//! in-ISO path (e.g. `/casper/vmlinuz`), locate the file and return a
//! `(start_byte, length_bytes)` pair so the HTTP handler can stream just
//! that range from the on-disk ISO without full extraction.
//!
//! We only implement what we need: the Primary Volume Descriptor and Rock
//! Ridge / Joliet extensions are ignored. Paths are matched case-insensitive
//! against plain ISO9660 filenames (uppercase, `;1` version suffix stripped).
//! This is sufficient for the kernel/initrd and wimboot files we serve;
//! if a requested path isn't found, the handler returns 404 and the user
//! can still download the whole ISO via `/iso/<id>.iso`.
use std::io::{Read, Seek, SeekFrom};
use std::path::Path;
const SECTOR: u64 = 2048;
#[derive(Debug, Clone)]
pub struct FileLocation {
pub offset: u64,
pub length: u64,
}
/// Look up `in_iso_path` (leading slash optional, case-insensitive) in the
/// ISO at `iso_path`. Returns None on any parsing or IO failure.
pub fn lookup(iso_path: &Path, in_iso_path: &str) -> Option<FileLocation> {
let mut f = std::fs::File::open(iso_path).ok()?;
let root = read_root_directory(&mut f)?;
let components: Vec<&str> = in_iso_path
.trim_start_matches('/')
.split('/')
.filter(|c| !c.is_empty())
.collect();
if components.is_empty() {
return None;
}
walk(&mut f, root.offset, root.length, &components)
}
fn read_root_directory(f: &mut std::fs::File) -> Option<FileLocation> {
// Primary Volume Descriptor at LBA 16.
let mut pvd = [0u8; 2048];
f.seek(SeekFrom::Start(16 * SECTOR)).ok()?;
f.read_exact(&mut pvd).ok()?;
if pvd[0] != 0x01 || &pvd[1..6] != b"CD001" {
return None;
}
// Root directory record is at offset 156, length 34.
let rec = &pvd[156..156 + 34];
let (offset, length) = parse_dir_record_ext(rec)?;
Some(FileLocation {
offset: offset * SECTOR,
length,
})
}
/// Walk components down the directory tree starting at `dir_offset`.
fn walk(
f: &mut std::fs::File,
dir_offset: u64,
dir_len: u64,
components: &[&str],
) -> Option<FileLocation> {
let mut dir = vec![0u8; dir_len as usize];
f.seek(SeekFrom::Start(dir_offset)).ok()?;
f.read_exact(&mut dir).ok()?;
let target = components[0];
let rest = &components[1..];
let mut i = 0;
while i < dir.len() {
let len = dir[i] as usize;
if len == 0 {
// Padding to sector boundary.
let next = (i / SECTOR as usize + 1) * SECTOR as usize;
if next <= i {
break;
}
i = next;
continue;
}
if i + len > dir.len() {
break;
}
let rec = &dir[i..i + len];
let name = dir_record_name(rec);
let is_dir = (rec.get(25).copied().unwrap_or(0) & 0x02) != 0;
// Skip "." (0x00) and ".." (0x01) pseudo-entries.
let is_pseudo = matches!(rec.get(32).copied(), Some(1))
&& rec.get(33).copied() == Some(0x00)
|| matches!(rec.get(32).copied(), Some(1)) && rec.get(33).copied() == Some(0x01);
if !is_pseudo && name.eq_ignore_ascii_case(target) {
let (child_off, child_len) = parse_dir_record_ext(rec)?;
if rest.is_empty() && !is_dir {
return Some(FileLocation {
offset: child_off * SECTOR,
length: child_len,
});
} else if !rest.is_empty() && is_dir {
return walk(f, child_off * SECTOR, child_len, rest);
}
}
i += len;
}
None
}
/// Extract (extent LBA, data length in bytes) from a directory record.
/// Layout per ISO9660: bytes 2..10 extent LBA (LE+BE duplicate), 10..18
/// data length (LE+BE duplicate). We trust the little-endian copy.
fn parse_dir_record_ext(rec: &[u8]) -> Option<(u64, u64)> {
if rec.len() < 34 {
return None;
}
let lba = u32::from_le_bytes(rec[2..6].try_into().ok()?) as u64;
let len = u32::from_le_bytes(rec[10..14].try_into().ok()?) as u64;
Some((lba, len))
}
/// Extract the identifier from a directory record, stripping ISO9660's
/// `;1` version suffix.
fn dir_record_name(rec: &[u8]) -> String {
let name_len = *rec.get(32).unwrap_or(&0) as usize;
if name_len == 0 || rec.len() < 33 + name_len {
return String::new();
}
let raw = &rec[33..33 + name_len];
let s = String::from_utf8_lossy(raw).to_string();
// Strip `;N` version suffix.
if let Some(i) = s.rfind(';') {
s[..i].to_string()
} else {
s
}
}
+3 -3
View File
@@ -9,8 +9,9 @@
//! and Linux kernel/initrd, without having to //! and Linux kernel/initrd, without having to
//! re-extract on every request) //! re-extract on every request)
//! //!
//! The `<id>/<path>` handler uses a read-only ISO9660 shim (see `iso_fs`) //! The `<id>/<path>` handler uses the read-only ISO9660 walker from
//! that lseeks into the ISO on disk — so we never keep extracted copies. //! `openpxe_iso_store::iso_fs` — seeking into the image wherever it
//! lives (local disk, NFS, SFTP), so we never keep extracted copies.
#![forbid(unsafe_code)] #![forbid(unsafe_code)]
pub mod app; pub mod app;
@@ -18,7 +19,6 @@ pub mod auth;
pub mod error; pub mod error;
pub mod grub_script; pub mod grub_script;
pub mod ipxe_script; pub mod ipxe_script;
pub mod iso_fs;
pub mod log_stream; pub mod log_stream;
pub mod notify; pub mod notify;
pub mod saml_routes; pub mod saml_routes;
+10 -1
View File
@@ -2,7 +2,7 @@ use crate::auth::SessionStore;
use crate::saml_routes::SamlRuntime; use crate::saml_routes::SamlRuntime;
use crate::uploads::UploadSessions; use crate::uploads::UploadSessions;
use openpxe_core::{ use openpxe_core::{
AdminStore, BootLog, BootRulesStore, BootTokens, BrandingStore, ClientRegistry, AdminStore, ApiKeyStore, BootLog, BootRulesStore, BootTokens, BrandingStore, ClientRegistry,
DeploymentQueue, HostBindings, LogBus, Metrics, NotifyStore, SettingsStore, SsoStore, DeploymentQueue, HostBindings, LogBus, Metrics, NotifyStore, SettingsStore, SsoStore,
}; };
use openpxe_iso_store::{ use openpxe_iso_store::{
@@ -55,6 +55,11 @@ pub struct AppState {
/// process restart (sessions are tied to UI state, not persisted — /// process restart (sessions are tied to UI state, not persisted —
/// matches Sonarr/Radarr behaviour). /// matches Sonarr/Radarr behaviour).
pub sessions: SessionStore, pub sessions: SessionStore,
/// v0.8.0: persisted operator API key. A request carrying a matching
/// `x-api-key` header authenticates exactly like an operator session,
/// so scripts / Postman can drive `/api/*` without a browser login.
/// Generated on first run; regenerable from Settings → Advanced.
pub api_key: ApiKeyStore,
/// SAML SSO configuration (persisted IdP metadata, Entity ID, toggles). /// SAML SSO configuration (persisted IdP metadata, Entity ID, toggles).
pub sso: SsoStore, pub sso: SsoStore,
/// v0.5.1: in-memory SAML runtime state — outstanding AuthnRequest IDs /// v0.5.1: in-memory SAML runtime state — outstanding AuthnRequest IDs
@@ -116,6 +121,10 @@ pub struct AppState {
/// `enp1s0`). Surfaced read-only on the Network tab. Empty if the /// `enp1s0`). Surfaced read-only on the Network tab. Empty if the
/// interface couldn't be identified. /// interface couldn't be identified.
pub nic_name: String, pub nic_name: String,
/// v0.7.2: physical link summary for that NIC (operstate, speed,
/// duplex, port MAC) — read from sysfs at startup; empty where
/// unavailable. Helps confirm which port answers PXE.
pub nic_link: String,
/// Subnet mask of the public interface in dotted-quad form. /// Subnet mask of the public interface in dotted-quad form.
pub subnet_mask: String, pub subnet_mask: String,
/// Default gateway IPv4 address. /// Default gateway IPv4 address.
+90 -19
View File
@@ -18,23 +18,16 @@ use openpxe_iso_store::{IsoStore, NfsShareManager, SftpShareManager, SmbShareMan
use tempfile::tempdir; use tempfile::tempdir;
use tower::ServiceExt; use tower::ServiceExt;
/// Build a tiny valid ISO9660 blob with volume label "ALPINE-TEST" so /// Build a tiny Alpine-shaped ISO9660 image: volume label "ALPINE-TEST"
/// introspection identifies it as Alpine. /// plus the real `/boot/vmlinuz-lts` + `/boot/initramfs-lts` tree.
/// v0.7.4's probe-based introspection verifies those paths exist before
/// emitting a kernel boot entry — a label-only blob no longer counts.
fn fake_alpine_iso() -> Vec<u8> { fn fake_alpine_iso() -> Vec<u8> {
let mut buf = vec![0u8; 32 * 2048]; openpxe_iso_store::iso_fs::testiso::TestIsoBuilder::new("ALPINE-TEST")
let off = 16 * 2048; .el_torito(true)
buf[off] = 0x01; .file("/boot/vmlinuz-lts", b"fake-kernel-bytes")
buf[off + 1..off + 6].copy_from_slice(b"CD001"); .file("/boot/initramfs-lts", b"fake-initramfs-bytes")
buf[off + 6] = 0x01; .build()
let label = b"ALPINE-TEST".to_vec();
let mut padded = label.clone();
padded.resize(32, b' ');
buf[off + 40..off + 40 + 32].copy_from_slice(&padded);
let term = 17 * 2048;
buf[term] = 0xFF;
buf[term + 1..term + 6].copy_from_slice(b"CD001");
buf[term + 6] = 0x01;
buf
} }
fn multipart_iso_body(filename: &str, bytes: &[u8]) -> (String, Vec<u8>) { fn multipart_iso_body(filename: &str, bytes: &[u8]) -> (String, Vec<u8>) {
@@ -106,6 +99,7 @@ async fn build_state() -> (AppState, tempfile::TempDir) {
let admin = openpxe_core::AdminStore::load_or_default(dir.path()); let admin = openpxe_core::AdminStore::load_or_default(dir.path());
let sso = openpxe_core::SsoStore::load_or_default(dir.path()); let sso = openpxe_core::SsoStore::load_or_default(dir.path());
let notify = openpxe_core::NotifyStore::load_or_default(dir.path()); let notify = openpxe_core::NotifyStore::load_or_default(dir.path());
let api_key = openpxe_core::ApiKeyStore::load_or_init(dir.path());
let sessions = openpxe_http_api::auth::SessionStore::default(); let sessions = openpxe_http_api::auth::SessionStore::default();
let metrics = Metrics::new(); let metrics = Metrics::new();
let state = AppState { let state = AppState {
@@ -121,6 +115,7 @@ async fn build_state() -> (AppState, tempfile::TempDir) {
pxe_bg_cache: openpxe_http_api::state::PxeBgCache::default(), pxe_bg_cache: openpxe_http_api::state::PxeBgCache::default(),
admin, admin,
sessions, sessions,
api_key,
sso, sso,
saml: openpxe_http_api::saml_routes::SamlRuntime::default(), saml: openpxe_http_api::saml_routes::SamlRuntime::default(),
notify, notify,
@@ -135,12 +130,67 @@ async fn build_state() -> (AppState, tempfile::TempDir) {
started_at: time::OffsetDateTime::now_utc(), started_at: time::OffsetDateTime::now_utc(),
public_base_url: "http://127.0.0.1".into(), public_base_url: "http://127.0.0.1".into(),
nic_name: "lo".into(), nic_name: "lo".into(),
nic_link: String::new(),
subnet_mask: "255.0.0.0".into(), subnet_mask: "255.0.0.0".into(),
gateway: "127.0.0.1".into(), gateway: "127.0.0.1".into(),
}; };
(state, dir) (state, dir)
} }
#[tokio::test]
async fn api_key_authenticates_gated_endpoints() {
// v0.8.0: the x-api-key header authenticates /api/* like an operator
// session. The middleware only enforces once an admin is configured
// (before that everything is open), so bootstrap one first.
let (state, _dir) = build_state().await;
state
.admin
.bootstrap("admin", "correct-horse-battery-staple")
.unwrap();
let key = state.api_key.current();
let app = build_router(state);
// No credentials → 401.
let res = app
.clone()
.oneshot(
Request::builder()
.uri("/api/isos")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(res.status(), StatusCode::UNAUTHORIZED, "no auth must 401");
// Wrong key → 401.
let res = app
.clone()
.oneshot(
Request::builder()
.uri("/api/isos")
.header("x-api-key", "not-the-key")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(res.status(), StatusCode::UNAUTHORIZED, "wrong key must 401");
// Correct key → 200 (operator-equivalent access).
let res = app
.oneshot(
Request::builder()
.uri("/api/isos")
.header("x-api-key", key)
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(res.status(), StatusCode::OK, "valid key must authenticate");
}
#[tokio::test] #[tokio::test]
async fn health_and_ready_endpoints() { async fn health_and_ready_endpoints() {
let (state, _dir) = build_state().await; let (state, _dir) = build_state().await;
@@ -1247,9 +1297,10 @@ async fn chunked_upload_writes_progressively_and_finishes_iso() {
.method("POST") .method("POST")
.uri("/api/uploads") .uri("/api/uploads")
.header("content-type", "application/json") .header("content-type", "application/json")
.body(Body::from( .body(Body::from(format!(
r#"{"filename":"chunked-alpine.iso","size_bytes":65536}"#, r#"{{"filename":"chunked-alpine.iso","size_bytes":{}}}"#,
)) iso.len()
)))
.unwrap(), .unwrap(),
) )
.await .await
@@ -2751,3 +2802,23 @@ async fn arch_selective_rule_ignores_other_arches() {
assert_eq!(s, StatusCode::OK); assert_eq!(s, StatusCode::OK);
assert!(String::from_utf8_lossy(&b).contains("boot rule -> fake-alpine-linux")); assert!(String::from_utf8_lossy(&b).contains("boot rule -> fake-alpine-linux"));
} }
#[tokio::test]
async fn boot_rule_driver_mode_pin_round_trips_via_api() {
// v0.7.1: a rule may pin only a boot binary (no target) — the API
// must persist and return it for the DHCP proxy to consult.
let (state, _dir) = build_state().await;
let app = build_router(state.clone());
let cfg = r#"{"rules":[{"mac_prefix":"aa:bb:cc","arch":"","target":"","driver_mode":"shim","enabled":true,"note":"SB rack"}],"webhook_url":""}"#;
let (s, _) = put_json(&app, "/api/boot-rules", cfg).await;
assert_eq!(s, StatusCode::NO_CONTENT);
let (s, b) = get(&app, "/api/boot-rules").await;
assert_eq!(s, StatusCode::OK);
let v: serde_json::Value = serde_json::from_slice(&b).unwrap();
assert_eq!(v["rules"][0]["driver_mode"], "shim");
// And the store the DHCP proxy shares resolves the pin.
assert_eq!(
state.boot_rules.driver_mode_hint("aa:bb:cc:00:00:07", None),
Some(openpxe_core::DriverMode::Shim)
);
}
-1
View File
@@ -13,4 +13,3 @@ workspace = true
openpxe-core.workspace = true openpxe-core.workspace = true
rust-embed.workspace = true rust-embed.workspace = true
tracing.workspace = true tracing.workspace = true
thiserror.workspace = true
+8 -3
View File
@@ -12,16 +12,13 @@ workspace = true
[dependencies] [dependencies]
openpxe-core.workspace = true openpxe-core.workspace = true
tokio = { workspace = true } tokio = { workspace = true }
tokio-util = { workspace = true }
serde.workspace = true serde.workspace = true
serde_json.workspace = true serde_json.workspace = true
tracing.workspace = true tracing.workspace = true
thiserror.workspace = true
anyhow.workspace = true anyhow.workspace = true
sha2.workspace = true sha2.workspace = true
hex.workspace = true hex.workspace = true
bcrypt.workspace = true bcrypt.workspace = true
uuid.workspace = true
time.workspace = true time.workspace = true
parking_lot.workspace = true parking_lot.workspace = true
bytes.workspace = true bytes.workspace = true
@@ -49,3 +46,11 @@ futures = { workspace = true }
[dev-dependencies] [dev-dependencies]
tempfile = "3.12" tempfile = "3.12"
[features]
# v0.7.4: exposes the in-memory ISO9660 test-image builder
# (`iso_fs::testiso`) to other crates' integration tests, so http-api's
# full-flow tests can synthesize ISOs with real directory trees — the
# probe-based introspection no longer classifies label-only blobs.
# Never enabled in production builds.
test-image = []
+613 -188
View File
@@ -1,16 +1,33 @@
//! ISO introspection — identify the distro family and locate kernel/initrd. //! ISO introspection — identify the distro family and locate kernel/initrd.
//! //!
//! We avoid a full ISO9660/Joliet/Rock-Ridge parser by reading a small number //! v0.7.4 rewrite: detection is **probe-based**. Instead of grepping raw
//! of well-known files via `isoinfo` (from cdrtools/genisoimage) when it's on //! sectors for filename strings (which false-positived — any Linux ISO
//! the path. As a pure-Rust fallback we do a crude scan: read the volume //! shipping GRUB/syslinux chainload modules contains the literal
//! descriptor at offset 0x8000 to grab the volume label, and grep for known //! "bootmgr", so gparted-live classified as Windows), we walk the
//! filenames by scanning raw sectors — good enough to tell Debian from RHEL //! ISO9660 directory tree via [`crate::iso_fs`] and check whether the
//! most of the time, without shelling out. //! well-known boot files actually exist. The same probes run over local
//! files and remote NFS/SFTP shares — remote ISOs finally classify
//! instead of registering as `Unknown`.
//! //!
//! The returned `IntrospectionReport` is what `BootEntry`s get generated from. //! Layered, first-decisive-answer-wins:
//! 1. PVD volume label → family hint.
//! 2. El Torito boot-catalog presence (the "bootable at all" signal).
//! 3. `/sources/boot.wim` directory probe → Windows install media.
//! 4. Linux probe table → verified kernel+initrd paths. A probe match
//! both classifies the family and (for the families whose boot
//! arguments we render) yields kernel paths that are *known to
//! exist* — no more guessed paths that 404 at boot.
//! 5. Bulk byte scan for UDF Windows markers — local images only
//! (modern Windows ISOs hide their tree from ISO9660; remote scans
//! skip this so a share rescan doesn't stream 16 MiB per ISO).
//! 6. Filename tokens — the last-resort hint, and the only signal
//! available for SMB shares (smbclient cannot seek).
//!
//! The returned `IntrospectionReport` is what `BootEntry`s get generated
//! from.
use crate::iso_fs::{self, CachingReadAt, FileReadAt, IsoReadAt, SECTOR};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::io::{Read, Seek, SeekFrom};
use std::path::Path; use std::path::Path;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)] #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)]
@@ -30,18 +47,35 @@ pub enum DistroFamily {
/// re-classify already-uploaded ISOs. On startup the store re-runs /// re-classify already-uploaded ISOs. On startup the store re-runs
/// `introspect` on any *local* ISO whose persisted report predates this /// `introspect` on any *local* ISO whose persisted report predates this
/// revision (see `IsoStore::load_from_disk`), so an upgrade fixes stale /// revision (see `IsoStore::load_from_disk`), so an upgrade fixes stale
/// metadata — e.g. a Windows 11 ISO tagged `Unknown` by an older binary — /// metadata without the operator having to delete and re-upload.
/// without the operator having to delete and re-upload it.
/// ///
/// rev 1 (v0.5.9): added El Torito boot-catalog detection + broadened /// rev 1 (v0.5.9): added El Torito boot-catalog detection + broadened
/// Windows (UDF/UTF-16) detection becomes retroactive. /// Windows (UDF/UTF-16) detection.
pub const INTROSPECT_REV: u32 = 1; /// rev 2 (v0.7.4): probe-based detection. Fixes Linux live ISOs that
/// classified as Windows via the raw "bootmgr" byte grep, verifies
/// kernel/initrd paths exist before emitting them, and adds the Debian
/// live / netinst / CoreOS shapes. Remote (NFS/SFTP) introspection
/// caches key off this rev too, so the cache self-invalidates.
/// rev 3 (v0.7.5): Joliet namespace fallback + gap-tolerant El Torito /
/// descriptor scans. Without this bump, images the rev-2 logic flagged
/// as data ISOs (mangled-primary appliance images, filler-sector boot
/// records) would never re-probe and stay mislabeled.
/// rev 4 (v0.8.0): dropped the over-broad "microsoft" UTF-16 bulk-scan
/// marker that classified any Secure-Boot-signed non-Windows bootable
/// (memtest86, signed BSDs, firmware tools) as Windows — the string
/// lives in the FAT long-filename entries of their MS-signed EFI loader.
/// The bump re-probes those so they drop the bogus Windows label.
pub const INTROSPECT_REV: u32 = 4;
#[derive(Debug, Clone, Default, Serialize, Deserialize)] #[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct IntrospectionReport { pub struct IntrospectionReport {
pub family: DistroFamily, pub family: DistroFamily,
pub volume_label: Option<String>, pub volume_label: Option<String>,
/// Kernel path inside the ISO (e.g. `/casper/vmlinuz`, `/isolinux/vmlinuz`). /// Kernel path inside the ISO (e.g. `/casper/vmlinuz`). v0.7.4: only
/// set when the path was verified to exist *and* the family's boot
/// arguments are known-good for direct kernel boot; families we can
/// only classify (Debian live, CoreOS live) leave it `None` so the
/// entry generator falls back to sanboot instead of a broken boot.
pub kernel_path: Option<String>, pub kernel_path: Option<String>,
/// Initrd path(s) inside the ISO. May be multiple for multi-initrd setups. /// Initrd path(s) inside the ISO. May be multiple for multi-initrd setups.
pub initrd_paths: Vec<String>, pub initrd_paths: Vec<String>,
@@ -55,124 +89,245 @@ pub struct IntrospectionReport {
/// appliance bundle) has no boot catalog and reports `false`. v0.5.9. /// appliance bundle) has no boot catalog and reports `false`. v0.5.9.
#[serde(default)] #[serde(default)]
pub el_torito: bool, pub el_torito: bool,
/// Revision of the introspection logic that produced this report. Old /// Revision of the introspection logic that produced this report.
/// `meta.json` files without the field deserialize as 0, which is /// `0` means "never introspected" (pre-v0.5.9 metadata, or a remote
/// below [`INTROSPECT_REV`], triggering a one-time re-introspect on /// ISO whose probe hasn't run / can't run) — the entry generator
/// the next startup. v0.5.9. /// treats those optimistically (sanboot) and the UI labels them.
#[serde(default)] #[serde(default)]
pub introspect_rev: u32, pub introspect_rev: u32,
} }
/// Probe an ISO file on disk. Never fails — on unrecoverable IO error we log /// One row of the Linux detection table.
/// and return an `Unknown` family so the uploader still sees a record. ///
/// `emit_kernel` distinguishes "we can boot this directly" from "we can
/// only classify it". Families marked `false` have boot protocols our
/// cmdline renderer doesn't speak yet (Debian-live `boot=live fetch=`,
/// d-i netinst, CoreOS `coreos.live.rootfs_url=`) — for those the probe
/// sets the family for the UI/menu but leaves `kernel_path` unset so the
/// ISO keeps its (working) sanboot entry instead of gaining a broken
/// kernel one. Strictly fewer broken boots than guessing.
struct LinuxProbe {
family: DistroFamily,
kernel: &'static str,
initrd_candidates: &'static [&'static str],
emit_kernel: bool,
}
const LINUX_PROBES: &[LinuxProbe] = &[
// Ubuntu and friends (casper) — the classic direct-boot shape.
LinuxProbe {
family: DistroFamily::DebianUbuntu,
kernel: "/casper/vmlinuz",
initrd_candidates: &["/casper/initrd", "/casper/initrd.lz", "/casper/initrd.gz"],
emit_kernel: true,
},
// Debian-live derivatives: gparted-live, Clonezilla, Kali live, tails.
// Classification only — live-boot needs `boot=live fetch=<squashfs>`
// which we don't render yet; sanboot of these images works today.
LinuxProbe {
family: DistroFamily::DebianUbuntu,
kernel: "/live/vmlinuz",
initrd_candidates: &["/live/initrd.img", "/live/initrd"],
emit_kernel: false,
},
// Debian installer (netinst/DVD). Classification only for the same
// reason — d-i sanboots fine.
LinuxProbe {
family: DistroFamily::DebianUbuntu,
kernel: "/install.amd/vmlinuz",
initrd_candidates: &["/install.amd/initrd.gz"],
emit_kernel: false,
},
// Anaconda family: RHEL, CentOS, Alma, Rocky, Fedora — and their
// many derivatives (Cisco ISE, Nagios appliances, …). The CoreOS
// variant of this shape is special-cased after the table.
LinuxProbe {
family: DistroFamily::RhelFedora,
kernel: "/images/pxeboot/vmlinuz",
initrd_candidates: &["/images/pxeboot/initrd.img"],
emit_kernel: true,
},
LinuxProbe {
family: DistroFamily::OpenSuse,
kernel: "/boot/x86_64/loader/linux",
initrd_candidates: &["/boot/x86_64/loader/initrd"],
emit_kernel: true,
},
LinuxProbe {
family: DistroFamily::Arch,
kernel: "/arch/boot/x86_64/vmlinuz-linux",
initrd_candidates: &["/arch/boot/x86_64/initramfs-linux.img"],
emit_kernel: true,
},
LinuxProbe {
family: DistroFamily::Alpine,
kernel: "/boot/vmlinuz-lts",
initrd_candidates: &["/boot/initramfs-lts"],
emit_kernel: true,
},
];
/// CoreOS-style live images (RHCOS, FCOS, OpenShift agent ISOs) carry
/// the anaconda pxeboot layout *plus* a rootfs image. Direct kernel boot
/// of those requires `coreos.live.rootfs_url=` (and for agent ISOs, the
/// ignition config embedded in the ISO device) — neither of which a
/// plain `inst.repo=` cmdline provides. Their sanboot path works, so
/// they classify as RHEL-family but keep the sanboot entry.
const COREOS_ROOTFS: &str = "/images/pxeboot/rootfs.img";
/// Probe an ISO file on disk. Never fails — on unrecoverable IO error we
/// log and return an `Unknown` family so the uploader still sees a record.
pub fn introspect(path: &Path) -> IntrospectionReport { pub fn introspect(path: &Path) -> IntrospectionReport {
let filename = path
.file_name()
.map(|s| s.to_string_lossy().into_owned())
.unwrap_or_default();
let Ok(f) = std::fs::File::open(path) else {
tracing::warn!(target: "openpxe::iso", "cannot open ISO for introspection: {}", path.display());
return IntrospectionReport {
introspect_rev: INTROSPECT_REV,
..Default::default()
};
};
let len = f.metadata().map_or(0, |m| m.len());
// `FileReadAt` completes every read inline (no real awaits), so this
// light-weight block_on never parks; callers already run us on the
// blocking pool.
futures::executor::block_on(introspect_reader(
&mut FileReadAt::new(f),
len,
&filename,
true,
))
}
/// The detection core, generic over any random-access source. `total_len`
/// is the image size (every caller knows it — file metadata locally, the
/// share listing remotely) and bounds the bulk scan, since `IsoReadAt`
/// reads are exact-or-error. `filename` feeds the last-resort token
/// heuristics; `allow_bulk_scan` gates the 16 MiB UDF-Windows byte scan
/// (local files only — remote shares would stream that much per ISO per
/// rescan).
pub async fn introspect_reader<R: IsoReadAt + Send>(
r: &mut R,
total_len: u64,
filename: &str,
allow_bulk_scan: bool,
) -> IntrospectionReport {
let mut report = IntrospectionReport { let mut report = IntrospectionReport {
introspect_rev: INTROSPECT_REV, introspect_rev: INTROSPECT_REV,
..Default::default() ..Default::default()
}; };
let Ok(mut f) = std::fs::File::open(path) else { // Everything sector-shaped goes through one caching wrapper: the
tracing::warn!(target: "openpxe::iso", "cannot open ISO for introspection: {}", path.display()); // descriptor-set sectors are read once and shared between the label
return report; // scan, the El Torito walk, and the namespace-root lookups; the
}; // probe table's repeated root/subdirectory reads collapse the same
// way — over NFS/SFTP that's the difference between ~6 and ~60+
// round-trips per ISO.
{
let mut cr = CachingReadAt::new(r);
// ISO9660 Primary Volume Descriptor at LBA 16 (offset 0x8000), 2048 bytes. // ISO9660 Primary Volume Descriptor: bytes 40..72 are the volume
// Bytes 40..72 are the Volume Identifier (space-padded, d-characters). // identifier (space-padded). v0.7.5: located by scanning the
let mut pvd = [0u8; 2048]; // descriptor set (tolerating filler sectors) instead of assuming
if f.seek(SeekFrom::Start(0x8000)).is_ok() && f.read_exact(&mut pvd).is_ok() { // a pristine sector 16.
// Byte 0 must be 0x01 (primary descriptor), bytes 1..6 = "CD001". if let Some(pvd) = iso_fs::find_descriptor(&mut cr, false).await {
if pvd[0] == 0x01 && &pvd[1..6] == b"CD001" { let label = String::from_utf8_lossy(&pvd[40..72]).trim().to_string();
let label_raw = &pvd[40..72];
let label = String::from_utf8_lossy(label_raw).trim().to_string();
if !label.is_empty() { if !label.is_empty() {
report.volume_label = Some(label.clone());
report.family = family_from_label(&label); report.family = family_from_label(&label);
report.volume_label = Some(label);
}
}
report.el_torito = detect_el_torito(&mut cr).await;
if iso_fs::exists(&mut cr, "/sources/boot.wim").await {
report.has_boot_wim = true;
report.family = DistroFamily::WindowsPe;
} else {
for probe in LINUX_PROBES {
if !iso_fs::exists(&mut cr, probe.kernel).await {
continue;
}
let mut initrd = None;
for cand in probe.initrd_candidates {
if iso_fs::exists(&mut cr, cand).await {
initrd = Some((*cand).to_string());
break;
}
}
let Some(initrd) = initrd else { continue };
// Content beats label: a rebadged derivative (volume
// label "ISE-3.2") with the anaconda layout is
// RHEL-family no matter what the label says.
report.family = probe.family;
let coreos = probe.family == DistroFamily::RhelFedora
&& iso_fs::exists(&mut cr, COREOS_ROOTFS).await;
if probe.emit_kernel && !coreos {
report.kernel_path = Some(probe.kernel.to_string());
report.initrd_paths = vec![initrd];
}
break;
} }
} }
} }
// Does the ISO have an El Torito boot catalog? This is what decides // Modern Windows 10/11 ISOs are UDF — their tree is invisible to the
// whether an ISO we *can't* otherwise classify is bootable at all — // ISO9660 walk and the volume label is a cryptic Microsoft string.
// a bootable ISO sanboots; a data/appliance ISO (no catalog) can't. // Scan the first 16 MiB for well-known markers, ASCII and UTF-16LE.
report.el_torito = detect_el_torito(&mut f); // Runs after the Linux probes so a Linux ISO that *contains* the
// string "bootmgr" (GRUB/syslinux chainload modules do) has already
// Cheap content scan: read the first ~64 MiB, look for signature filenames. // classified and never reaches this — that ordering is the v0.7.4
// This is enough to identify `sources/boot.wim` (Windows) and common // gparted-misdetection fix.
// kernel/initrd paths for the major Linux distros. if report.family == DistroFamily::Unknown && allow_bulk_scan {
let _ = f.seek(SeekFrom::Start(0)); if let Some(win) = bulk_windows_scan(r, total_len).await {
let scan_bytes = 64 * 1024 * 1024;
let mut buf = vec![0u8; 1024 * 1024];
let mut read_total = 0usize;
// Size the haystack to what will actually be read — the scan cap or
// the file itself, whichever is smaller — so the fill never reallocs
// and a small ISO doesn't reserve the full 64 MiB.
let file_len = f.metadata().map_or(usize::MAX, |m| {
usize::try_from(m.len()).unwrap_or(usize::MAX)
});
let mut haystack = Vec::with_capacity(scan_bytes.min(file_len));
while read_total < scan_bytes {
let n = f.read(&mut buf).unwrap_or(0);
if n == 0 {
break;
}
haystack.extend_from_slice(&buf[..n]);
read_total += n;
}
// `sources/boot.wim` is the definitive Windows-install-media marker
// when the ISO exposes ASCII (ISO9660/Joliet) names. `contains_ascii`
// is case-insensitive, so one form covers BOOT.WIM / boot.wim and the
// backslash variant.
if contains_ascii(&haystack, b"sources/boot.wim")
|| contains_ascii(&haystack, b"sources\\boot.wim")
{
report.has_boot_wim = true;
report.family = DistroFamily::WindowsPe;
}
// v0.5.8: broaden Windows detection. Modern Windows 10/11 ISOs are
// UDF — filenames are stored as UTF-16 (so the ASCII scan above misses
// them) and the volume label is a cryptic Microsoft string (so
// `family_from_label` misses it too). Booting is via HTTP sanboot of
// the raw ISO (no boot.wim extraction), so we only need the *family*.
// Catch the common cases: well-known Windows markers in either ASCII
// or UTF-16LE within the first 16 MiB, plus a filename hint.
if report.family == DistroFamily::Unknown {
let head = &haystack[..haystack.len().min(16 * 1024 * 1024)];
let ascii_markers: [&[u8]; 4] = [
b"bootmgr",
b"sources/install.wim",
b"sources/install.esd",
b"efi/microsoft",
];
let utf16_markers = ["bootmgr", "boot.wim", "install.wim", "microsoft"];
let looks_windows = ascii_markers.iter().any(|m| contains_ascii(head, m))
|| utf16_markers.iter().any(|m| contains_utf16le_ci(head, m))
|| filename_looks_windows(path);
if looks_windows {
report.family = DistroFamily::WindowsPe; report.family = DistroFamily::WindowsPe;
report.has_boot_wim = win;
} }
} }
// Best-effort kernel/initrd path guess from family. These paths are what // Last resort: filename tokens. The only signal for SMB-sourced ISOs
// distro ISOs conventionally ship at — we don't verify extraction here; // and renamed/UDF images that defeated everything above.
// that happens in the store after introspection. if report.family == DistroFamily::Unknown {
let (k, i) = guess_kernel_initrd(report.family); report.family = family_from_filename(filename);
report.kernel_path = k.map(str::to_string); }
report.initrd_paths = i.iter().map(std::string::ToString::to_string).collect();
report report
} }
/// Provisional report for a remote ISO that hasn't been (or can't be)
/// content-probed yet: family from the filename, `introspect_rev` left
/// at 0 so the entry generator keeps the optimistic sanboot entry and
/// the UI shows it as awaiting introspection. Used by all three share
/// managers at registration; NFS/SFTP upgrade it in the background.
#[must_use]
pub fn provisional_report(filename: &str) -> IntrospectionReport {
IntrospectionReport {
family: family_from_filename(filename),
..Default::default()
}
}
fn family_from_label(label: &str) -> DistroFamily { fn family_from_label(label: &str) -> DistroFamily {
let l = label.to_ascii_lowercase(); let l = label.to_ascii_lowercase();
if l.contains("ubuntu") || l.contains("debian") || l.contains("mint") { if l.contains("ubuntu")
|| l.contains("debian")
|| l.contains("mint")
|| l.contains("kali")
|| l.contains("gparted")
|| l.contains("clonezilla")
{
DistroFamily::DebianUbuntu DistroFamily::DebianUbuntu
} else if l.contains("rhel") } else if l.contains("rhel")
|| l.contains("centos") || l.contains("centos")
|| l.contains("fedora") || l.contains("fedora")
|| l.contains("rocky") || l.contains("rocky")
|| l.contains("alma") || l.contains("alma")
|| l.contains("rhcos")
|| l.contains("coreos")
|| l.contains("openshift")
|| l.contains("okd")
{ {
DistroFamily::RhelFedora DistroFamily::RhelFedora
} else if l.contains("suse") || l.contains("opensuse") { } else if l.contains("suse") || l.contains("opensuse") {
@@ -188,23 +343,97 @@ fn family_from_label(label: &str) -> DistroFamily {
} }
} }
fn guess_kernel_initrd(family: DistroFamily) -> (Option<&'static str>, Vec<&'static str>) { /// Filename token heuristic — `AlmaLinux-9.5-x86_64-dvd.iso` says what
match family { /// it is even when we can't read a byte of it. Tokens are the filename
DistroFamily::DebianUbuntu => (Some("/casper/vmlinuz"), vec!["/casper/initrd"]), /// split on every non-alphanumeric character, so "almalinux", "rhel",
DistroFamily::RhelFedora => ( /// "win11" match without "search" tripping the "arch" token.
Some("/images/pxeboot/vmlinuz"), pub fn family_from_filename(filename: &str) -> DistroFamily {
vec!["/images/pxeboot/initrd.img"], if filename_looks_windows(filename) {
), return DistroFamily::WindowsPe;
DistroFamily::OpenSuse => ( }
Some("/boot/x86_64/loader/linux"), let lower = filename.to_ascii_lowercase();
vec!["/boot/x86_64/loader/initrd"], let tokens: Vec<&str> = lower
), .split(|c: char| !c.is_ascii_alphanumeric())
DistroFamily::Arch => ( .filter(|t| !t.is_empty())
Some("/arch/boot/x86_64/vmlinuz-linux"), .collect();
vec!["/arch/boot/x86_64/initramfs-linux.img"], let has = |t: &str| tokens.contains(&t);
), if has("ubuntu")
DistroFamily::Alpine => (Some("/boot/vmlinuz-lts"), vec!["/boot/initramfs-lts"]), || has("debian")
DistroFamily::WindowsPe | DistroFamily::Unknown => (None, Vec::new()), || has("mint")
|| has("kali")
|| has("gparted")
|| has("clonezilla")
|| has("tails")
{
DistroFamily::DebianUbuntu
} else if has("rhel")
|| has("centos")
|| has("almalinux")
|| has("alma")
|| has("rocky")
|| has("rockylinux")
|| has("fedora")
|| has("rhcos")
|| has("coreos")
|| has("openshift")
|| has("okd")
{
DistroFamily::RhelFedora
} else if has("opensuse") || has("suse") || has("sles") {
DistroFamily::OpenSuse
} else if has("arch") || has("archlinux") || has("manjaro") {
DistroFamily::Arch
} else if has("alpine") {
DistroFamily::Alpine
} else {
DistroFamily::Unknown
}
}
/// Scan the first 16 MiB (or the whole image when smaller) for Windows
/// markers. Returns `Some(has_boot_wim)` on a hit, `None` when nothing
/// Windows-shaped is found.
async fn bulk_windows_scan<R: IsoReadAt + Send>(r: &mut R, total_len: u64) -> Option<bool> {
const SCAN_BYTES: u64 = 16 * 1024 * 1024;
const CHUNK: u64 = 1024 * 1024;
let budget = SCAN_BYTES.min(total_len);
let mut haystack = Vec::with_capacity(usize::try_from(budget).unwrap_or(0));
let mut offset = 0u64;
while offset < budget {
// Reads are exact-or-error, so clamp the final chunk to what the
// image actually has — netboot.xyz is 2.3 MB, not 16.
let want = u32::try_from(CHUNK.min(budget - offset)).unwrap_or(u32::MAX);
let Ok(chunk) = r.read_at(offset, want).await else {
break; // read error: scan what we have
};
offset += chunk.len() as u64;
haystack.extend_from_slice(&chunk);
}
if haystack.is_empty() {
return None;
}
let boot_wim = contains_ascii(&haystack, b"sources/boot.wim")
|| contains_ascii(&haystack, b"sources\\boot.wim")
|| contains_utf16le_ci(&haystack, "boot.wim");
if boot_wim {
return Some(true);
}
let ascii_markers: [&[u8]; 3] = [b"bootmgr", b"sources/install.wim", b"sources/install.esd"];
// v0.8.0: dropped the bare "microsoft" marker. It matched the
// Microsoft-signed Secure-Boot EFI loader that memtest86 (and signed
// BSDs / firmware tools) ship — the string lives in the loader's FAT
// long-filename entries — so any signed non-Windows bootable
// false-classified as Windows. The remaining markers are all
// Windows-exclusive filenames.
let utf16_markers = ["bootmgr", "install.wim"];
let hit = ascii_markers.iter().any(|m| contains_ascii(&haystack, m))
|| utf16_markers
.iter()
.any(|m| contains_utf16le_ci(&haystack, m));
if hit {
Some(false)
} else {
None
} }
} }
@@ -237,14 +466,10 @@ fn contains_utf16le_ci(haystack: &[u8], ascii: &str) -> bool {
/// Filename heuristic: a stock Windows ISO almost always carries an obvious /// Filename heuristic: a stock Windows ISO almost always carries an obvious
/// token in its name (e.g. `..._windows_11_...`, `Win10`, `winserver`). /// token in its name (e.g. `..._windows_11_...`, `Win10`, `winserver`).
/// Used only as a last-resort family hint when the content scan and volume /// v0.7.4: takes the bare filename instead of a `Path` so the same check
/// label are inconclusive. v0.5.8. /// runs against remote share listings.
fn filename_looks_windows(path: &Path) -> bool { fn filename_looks_windows(filename: &str) -> bool {
let name = path let name = filename.to_ascii_lowercase();
.file_name()
.and_then(|s| s.to_str())
.unwrap_or("")
.to_ascii_lowercase();
const TOKENS: [&str; 6] = [ const TOKENS: [&str; 6] = [
"windows", "windows",
"winpe", "winpe",
@@ -263,32 +488,32 @@ const EL_TORITO_ID: &[u8] = b"EL TORITO SPECIFICATION";
/// Detect an El Torito boot catalog — the marker that an ISO is bootable /// Detect an El Torito boot catalog — the marker that an ISO is bootable
/// by BIOS/UEFI firmware (and thus by iPXE `sanboot`). /// by BIOS/UEFI firmware (and thus by iPXE `sanboot`).
/// ///
/// The ISO9660 Volume Descriptor Set starts at LBA 16 (offset 0x8000) and /// The ISO9660 Volume Descriptor Set starts at LBA 16 and runs one
/// runs one 2048-byte descriptor per sector until a Set Terminator /// 2048-byte descriptor per sector; a Boot Record descriptor (type 0x00)
/// (type 0xFF). A Boot Record descriptor (type 0x00) whose 32-byte boot /// whose 32-byte boot system identifier reads "EL TORITO SPECIFICATION"
/// system identifier reads "EL TORITO SPECIFICATION" means the image /// means the image declares a boot catalog. We only confirm its presence
/// declares an El Torito boot catalog. We only confirm its presence — we /// — we don't parse the catalog (sanboot/the firmware does that).
/// don't parse the catalog (sanboot/the firmware does that). The walk is ///
/// capped so a malformed/huge image can't spin us. v0.5.9. /// v0.7.5: the walk no longer aborts at the first non-`CD001` sector or
fn detect_el_torito(f: &mut std::fs::File) -> bool { /// stops at a Set Terminator. Sloppy mastering tools (appliance ISOs
let mut vd = [0u8; 2048]; /// especially) leave zeroed filler sectors inside the descriptor area or
/// odd descriptor ordering, which used to hide a real boot record and
/// flag a bootable image as a data ISO. All 16 sectors are examined —
/// the signature is 25 exact bytes, so scanning past the terminator
/// (into e.g. a UDF volume recognition sequence) cannot false-positive.
/// The cap keeps a malformed image from spinning us. v0.5.9 originally;
/// reader-generic since v0.7.4.
async fn detect_el_torito<R: IsoReadAt + Send>(r: &mut R) -> bool {
for lba in 16u64..32 { for lba in 16u64..32 {
if f.seek(SeekFrom::Start(lba * 2048)).is_err() || f.read_exact(&mut vd).is_err() { let Ok(vd) = r.read_at(lba * SECTOR, 2048).await else {
// Past end of a tiny image — nothing more to examine.
return false; return false;
} };
// Every descriptor in the set carries the "CD001" magic; once it's
// missing we've walked off the end of a valid set.
if &vd[1..6] != b"CD001" { if &vd[1..6] != b"CD001" {
return false; continue; // filler/garbage sector — keep walking
} }
match vd[0] { if vd[0] == 0x00 && vd[7..7 + EL_TORITO_ID.len()] == *EL_TORITO_ID {
// Boot Record descriptor carrying the El Torito signature. return true;
0x00 if vd[7..7 + EL_TORITO_ID.len()] == *EL_TORITO_ID => return true,
// Volume Descriptor Set Terminator — nothing bootable found.
0xFF => return false,
// Any other descriptor (incl. a non-El-Torito boot record) —
// keep walking the set.
_ => {}
} }
} }
false false
@@ -297,6 +522,12 @@ fn detect_el_torito(f: &mut std::fs::File) -> bool {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::iso_fs::testiso::{MemReadAt, TestIsoBuilder};
fn introspect_mem(img: Vec<u8>, filename: &str, bulk: bool) -> IntrospectionReport {
let len = img.len() as u64;
futures::executor::block_on(introspect_reader(&mut MemReadAt(img), len, filename, bulk))
}
#[test] #[test]
fn label_matching() { fn label_matching() {
@@ -313,13 +544,158 @@ mod tests {
DistroFamily::OpenSuse DistroFamily::OpenSuse
); );
assert_eq!(family_from_label("ARCH_202604"), DistroFamily::Arch); assert_eq!(family_from_label("ARCH_202604"), DistroFamily::Arch);
assert_eq!(
family_from_label("GParted-live"),
DistroFamily::DebianUbuntu
);
assert_eq!(family_from_label("rhcos-417"), DistroFamily::RhelFedora);
assert_eq!(family_from_label("weird-custom"), DistroFamily::Unknown); assert_eq!(family_from_label("weird-custom"), DistroFamily::Unknown);
} }
#[test]
fn gparted_shape_is_not_windows() {
// The v0.7.4 regression test: a Debian-live image whose payload
// contains the literal string "bootmgr" (as GRUB/syslinux
// chainload modules do). The old byte-grep classified this as
// WindowsPe; the probe order must classify Debian first.
let img = TestIsoBuilder::new("GParted-live")
.el_torito(true)
.file("/live/vmlinuz", b"KERNEL")
.file("/live/initrd.img", b"INITRD")
.file("/boot/grub/chain.mod", b"xxx bootmgr xxx")
.build();
let r = introspect_mem(img, "gparted-live-1.8.1-3-amd64.iso", true);
assert_eq!(r.family, DistroFamily::DebianUbuntu);
// Classification only — live-boot args aren't rendered yet, so no
// kernel entry; sanboot (via el_torito) keeps working.
assert!(r.kernel_path.is_none());
assert!(r.el_torito);
assert_eq!(r.introspect_rev, INTROSPECT_REV);
}
#[test]
fn casper_shape_verifies_kernel_and_initrd() {
let img = TestIsoBuilder::new("Ubuntu-Server 24.04.1 LTS amd64")
.el_torito(true)
.file("/casper/vmlinuz", b"K")
.file("/casper/initrd", b"I")
.build();
let r = introspect_mem(img, "ubuntu-24.04.1-live-server-amd64.iso", true);
assert_eq!(r.family, DistroFamily::DebianUbuntu);
assert_eq!(r.kernel_path.as_deref(), Some("/casper/vmlinuz"));
assert_eq!(r.initrd_paths, vec!["/casper/initrd".to_string()]);
}
#[test]
fn anaconda_shape_emits_verified_paths() {
let img = TestIsoBuilder::new("AlmaLinux-9-5-x86_64-dvd")
.el_torito(true)
.file("/images/pxeboot/vmlinuz", b"K")
.file("/images/pxeboot/initrd.img", b"I")
.build();
let r = introspect_mem(img, "AlmaLinux-9.5-x86_64-dvd.iso", true);
assert_eq!(r.family, DistroFamily::RhelFedora);
assert_eq!(r.kernel_path.as_deref(), Some("/images/pxeboot/vmlinuz"));
}
#[test]
fn coreos_shape_classifies_but_keeps_sanboot() {
// RHCOS / OpenShift agent ISOs: anaconda layout + rootfs.img.
// Direct kernel boot needs coreos.live.rootfs_url (and agent
// ISOs their embedded ignition), so kernel_path must stay None.
let img = TestIsoBuilder::new("rhcos-417.94.202501")
.el_torito(true)
.file("/images/pxeboot/vmlinuz", b"K")
.file("/images/pxeboot/initrd.img", b"I")
.file("/images/pxeboot/rootfs.img", b"R")
.build();
let r = introspect_mem(img, "rhcos-live.x86_64.iso", true);
assert_eq!(r.family, DistroFamily::RhelFedora);
assert!(
r.kernel_path.is_none(),
"CoreOS must not get a kernel entry"
);
assert!(r.el_torito);
}
#[test]
fn boot_wim_probe_classifies_windows() {
let img = TestIsoBuilder::new("CCCOMA_X64FRE_EN-US_DV9")
.el_torito(true)
.file("/sources/boot.wim", b"MSWIMMSWIM")
.build();
let r = introspect_mem(img, "whatever.iso", false);
assert_eq!(r.family, DistroFamily::WindowsPe);
assert!(r.has_boot_wim);
}
#[test]
fn label_only_linux_without_verified_kernel_gets_no_kernel_path() {
// Label says RHEL but the tree has no pxeboot files — the old
// code guessed `/images/pxeboot/vmlinuz` and emitted an entry
// that 404'd at boot. Now: family yes, kernel paths no.
let img = TestIsoBuilder::new("RHEL-9-5-CUSTOM")
.el_torito(true)
.file("/readme.txt", b"hi")
.build();
let r = introspect_mem(img, "rhel-custom.iso", true);
assert_eq!(r.family, DistroFamily::RhelFedora);
assert!(r.kernel_path.is_none());
assert!(r.initrd_paths.is_empty());
}
#[test]
fn remote_skips_bulk_scan_but_filename_still_hints() {
// No ISO9660 signatures at all (e.g. pure-UDF image read over a
// share), bulk scan off: filename is the only signal.
let img = vec![0u8; 64 * 1024];
let r = introspect_mem(img.clone(), "Win11_24H2_English_x64.iso", false);
assert_eq!(r.family, DistroFamily::WindowsPe);
let r2 = introspect_mem(img, "mystery.iso", false);
assert_eq!(r2.family, DistroFamily::Unknown);
}
#[test]
fn filename_family_table() {
use DistroFamily::*;
let cases = [
("AlmaLinux-9.5-x86_64-dvd.iso", RhelFedora),
("CentOS-Stream-10-latest-x86_64-dvd1.iso", RhelFedora),
("rhel-9.0-x86_64-boot.iso", RhelFedora),
("rhcos-live.x86_64.iso", RhelFedora),
("openshift-4-21-9.agent.x86_64.iso", RhelFedora),
("ubuntu-24.04-desktop.iso", DebianUbuntu),
("gparted-live-1.8.1-3-amd64.iso", DebianUbuntu),
("archlinux-2026.05.01-x86_64.iso", Arch),
("arch-2026.05.01.iso", Arch),
("alpine-standard-3.21.0-x86_64.iso", Alpine),
("openSUSE-Leap-15.6-DVD-x86_64.iso", OpenSuse),
("en-us_windows_11_iot_enterprise.iso", WindowsPe),
("Win10_22H2_English_x64.iso", WindowsPe),
("netboot.xyz.iso", Unknown),
("ise-3.2.0.542a.SPA.x86_64.iso", Unknown),
("Macrium_5860_v2.iso", Unknown),
// "search" must not trip the "arch" token.
("research-data.iso", Unknown),
];
for (name, want) in cases {
assert_eq!(family_from_filename(name), want, "{name}");
}
}
#[test]
fn provisional_report_keeps_rev_zero() {
let r = provisional_report("rhel-9.0-x86_64-dvd.iso");
assert_eq!(r.family, DistroFamily::RhelFedora);
assert_eq!(
r.introspect_rev, 0,
"provisional must keep optimistic sanboot"
);
assert!(!r.el_torito);
}
#[test] #[test]
fn utf16le_marker_matches_case_insensitively() { fn utf16le_marker_matches_case_insensitively() {
// "boot.wim" encoded UTF-16LE, mixed case — UDF stores Windows
// filenames this way, which the ASCII scan can't see.
let s = "BOOT.WIM"; let s = "BOOT.WIM";
let utf16: Vec<u8> = s.bytes().flat_map(|b| [b, 0]).collect(); let utf16: Vec<u8> = s.bytes().flat_map(|b| [b, 0]).collect();
let mut hay = vec![0u8; 8]; let mut hay = vec![0u8; 8];
@@ -328,59 +704,108 @@ mod tests {
assert!(contains_utf16le_ci(&hay, "boot.wim")); assert!(contains_utf16le_ci(&hay, "boot.wim"));
assert!(contains_utf16le_ci(&hay, "Boot.Wim")); assert!(contains_utf16le_ci(&hay, "Boot.Wim"));
assert!(!contains_utf16le_ci(&hay, "install.wim")); assert!(!contains_utf16le_ci(&hay, "install.wim"));
// An ASCII (not UTF-16) occurrence must NOT match the UTF-16 scan.
assert!(!contains_utf16le_ci(b"boot.wim plain ascii", "boot.wim")); assert!(!contains_utf16le_ci(b"boot.wim plain ascii", "boot.wim"));
} }
#[test] #[test]
fn el_torito_boot_catalog_detected() { fn el_torito_detected_through_reader() {
let dir = tempfile::tempdir().unwrap(); let with = TestIsoBuilder::new("BOOTABLE").el_torito(true).build();
// Helper: stamp a 2048-byte descriptor at `lba` with type + magic. let without = TestIsoBuilder::new("DATA").build();
let stamp = |img: &mut [u8], lba: usize, ty: u8| { assert!(futures::executor::block_on(detect_el_torito(
let off = lba * 2048; &mut MemReadAt(with)
img[off] = ty; )));
img[off + 1..off + 6].copy_from_slice(b"CD001"); assert!(!futures::executor::block_on(detect_el_torito(
}; &mut MemReadAt(without)
)));
}
// Bootable image: PVD @16, El Torito Boot Record @17, terminator @18. #[test]
let mut boot = vec![0u8; 2048 * 19]; fn el_torito_survives_filler_sector_in_descriptor_area() {
stamp(&mut boot, 16, 0x01); // v0.7.5 tolerance test: sloppy appliance mastering leaves a
stamp(&mut boot, 17, 0x00); // zeroed sector inside the Volume Descriptor Set. The old walk
boot[17 * 2048 + 7..17 * 2048 + 7 + EL_TORITO_ID.len()].copy_from_slice(EL_TORITO_ID); // aborted at the first non-CD001 sector and flagged a genuinely
stamp(&mut boot, 18, 0xFF); // bootable image as a data ISO.
let bp = dir.path().join("boot.iso"); let sector = SECTOR as usize;
std::fs::write(&bp, &boot).unwrap(); let mut img = TestIsoBuilder::new("GAPPY").el_torito(true).build();
let mut f = std::fs::File::open(&bp).unwrap(); // Builder layout: PVD @16, Boot Record @17, terminator @18.
// Move the BR to 18 (over the terminator) and zero out 17.
img.copy_within(17 * sector..18 * sector, 18 * sector);
img[17 * sector..18 * sector].fill(0);
assert!( assert!(
detect_el_torito(&mut f), futures::executor::block_on(detect_el_torito(&mut MemReadAt(img))),
"El Torito boot record should match" "boot record behind a zeroed filler sector must still be found"
); );
}
// Data/appliance image: PVD @16, terminator @17, no boot record. #[test]
let mut data = vec![0u8; 2048 * 18]; fn joliet_only_image_classifies_via_fallback() {
stamp(&mut data, 16, 0x01); // Primary namespace bare, real tree only in Joliet — the v0.7.5
stamp(&mut data, 17, 0xFF); // fallback must classify it (boot.wim probe) where v0.7.4 saw
let dp = dir.path().join("data.iso"); // "no installer files".
std::fs::write(&dp, &data).unwrap(); let img = TestIsoBuilder::new("WIN_APPLIANCE")
let mut f2 = std::fs::File::open(&dp).unwrap(); .el_torito(true)
assert!(!detect_el_torito(&mut f2), "data ISO has no boot catalog"); .joliet_only(true)
.file("/sources/boot.wim", b"WIMWIM")
.build();
let r = introspect_mem(img, "appliance.iso", false);
assert_eq!(r.family, DistroFamily::WindowsPe);
assert!(r.has_boot_wim);
assert!(r.el_torito);
// Same for a Linux shape: verified kernel paths via Joliet.
let img2 = TestIsoBuilder::new("CUSTOM-EL9")
.el_torito(true)
.joliet_only(true)
.file("/images/pxeboot/vmlinuz", b"K")
.file("/images/pxeboot/initrd.img", b"I")
.build();
let r2 = introspect_mem(img2, "custom-el9.iso", false);
assert_eq!(r2.family, DistroFamily::RhelFedora);
assert_eq!(r2.kernel_path.as_deref(), Some("/images/pxeboot/vmlinuz"));
} }
#[test] #[test]
fn filename_hint_catches_windows_isos() { fn filename_hint_catches_windows_isos() {
use std::path::Path; assert!(filename_looks_windows(
assert!(filename_looks_windows(Path::new(
"en-us_windows_11_iot_enterprise_ltsc_2024_x64_dvd.iso" "en-us_windows_11_iot_enterprise_ltsc_2024_x64_dvd.iso"
))); ));
assert!(filename_looks_windows(Path::new( assert!(filename_looks_windows("Win10_22H2_English_x64.iso"));
"Win10_22H2_English_x64.iso" assert!(filename_looks_windows("winserver2022.iso"));
))); assert!(!filename_looks_windows("ubuntu-24.04-desktop.iso"));
assert!(filename_looks_windows(Path::new("winserver2022.iso"))); assert!(!filename_looks_windows("Rocky-9.4-x86_64-dvd.iso"));
assert!(!filename_looks_windows(Path::new( }
"ubuntu-24.04-desktop.iso"
))); #[test]
assert!(!filename_looks_windows(Path::new( fn bulk_scan_catches_udf_windows_markers() {
"Rocky-9.4-x86_64-dvd.iso" // A blob with no ISO9660 tree but a UTF-16 "install.wim" — the
))); // UDF Windows shape after every probe missed.
let mut img = vec![0u8; 256 * 1024];
let marker: Vec<u8> = "install.wim".bytes().flat_map(|b| [b, 0]).collect();
img[100_000..100_000 + marker.len()].copy_from_slice(&marker);
let r = introspect_mem(img, "renamed.iso", true);
assert_eq!(r.family, DistroFamily::WindowsPe);
assert!(!r.has_boot_wim);
}
#[test]
fn memtest_signed_efi_is_not_windows() {
// v0.8.0 regression: PassMark MemTest86 ships a Microsoft-signed
// Secure-Boot EFI loader, and "Microsoft" appears in its FAT
// long-filename entries as UTF-16LE. The old bulk-scan "microsoft"
// marker classified it (and any signed BSD / firmware tool) as
// Windows. It must now classify as a generic bootable (sanboot).
let mut img = TestIsoBuilder::new("MEMTEST86")
.el_torito(true)
.file("/EFI/BOOT/BOOTX64.EFI", b"signed-efi-app")
.build();
let marker: Vec<u8> = "Microsoft".bytes().flat_map(|b| [b, 0]).collect();
img.extend_from_slice(&marker);
let r = introspect_mem(img, "memtest86-iso.iso", true);
assert_ne!(
r.family,
DistroFamily::WindowsPe,
"a Microsoft-signed EFI loader is not Windows media"
);
assert!(r.el_torito, "still a bootable image");
} }
} }
+761
View File
@@ -0,0 +1,761 @@
//! Read-only ISO9660 lookup over any random-access byte source.
//!
//! v0.7.4: generalized from the http-api crate's local-file-only walker so
//! the same directory walk drives three consumers:
//!
//! 1. `iso_file` HTTP serving — locate `/casper/vmlinuz` inside a local
//! *or remote* (NFS/SFTP) ISO and stream just that byte range.
//! 2. Introspection — probe for well-known kernel/initrd/boot.wim paths
//! instead of grepping raw sectors for filename strings (which
//! false-positived: any Linux ISO shipping GRUB/syslinux chainload
//! modules contains the literal "bootmgr" and used to classify as
//! Windows).
//! 3. Remote introspection — the same probes over an NFSv3 READ3 /
//! SFTP seek-read connection, which is what finally classifies
//! share-sourced ISOs instead of registering them all as `Unknown`.
//!
//! Namespaces: the primary ISO9660 tree is tried first; on a miss the
//! walk falls back to the **Joliet** supplementary namespace (v0.7.5) —
//! Windows-oriented mastering tools often write a minimal/mangled
//! primary tree with the real names only in Joliet. Rock Ridge stays
//! ignored. Matching is case-insensitive with the `;1` version suffix
//! and the trailing dot of extension-less strict-mastered names
//! stripped.
use std::collections::HashMap;
use std::future::Future;
use std::io::{Read, Seek, SeekFrom};
use std::path::Path;
pub const SECTOR: u64 = 2048;
/// Upper bound on a single directory extent we'll buffer. Real distro ISO
/// directories are a handful of KiB; the cap keeps a malformed or hostile
/// image from asking us to allocate gigabytes.
const MAX_DIR_BYTES: u64 = 4 * 1024 * 1024;
/// Byte range of one file inside the ISO image.
#[derive(Debug, Clone)]
pub struct FileLocation {
pub offset: u64,
pub length: u64,
}
/// Random-access reads into an ISO image. Implemented by a local
/// `std::fs::File`, the NFS and SFTP share readers, and the in-memory
/// test image.
///
/// The contract is `read_exact`-like: the returned buffer is exactly
/// `len` bytes or the call errors. The future must be `Send` because
/// remote introspection runs inside spawned tokio tasks.
pub trait IsoReadAt {
fn read_at(
&mut self,
offset: u64,
len: u32,
) -> impl Future<Output = std::io::Result<Vec<u8>>> + Send;
}
/// Local-file reader. The reads are synchronous inside an async fn —
/// callers run it either on the blocking pool (introspection at upload)
/// or through [`lookup_local`]'s `block_on`, never on a hot runtime
/// worker with real awaits pending.
pub struct FileReadAt(std::fs::File);
impl FileReadAt {
#[must_use]
pub fn new(f: std::fs::File) -> Self {
Self(f)
}
}
impl IsoReadAt for FileReadAt {
async fn read_at(&mut self, offset: u64, len: u32) -> std::io::Result<Vec<u8>> {
self.0.seek(SeekFrom::Start(offset))?;
let mut buf = vec![0u8; len as usize];
self.0.read_exact(&mut buf)?;
Ok(buf)
}
}
/// Exact-key read cache for the probe phase of introspection. The probe
/// table looks up ~20 paths and every one of them re-reads the root
/// directory (and usually one shared subdirectory); over NFS/SFTP that
/// would be 20 identical round-trips. Directory reads repeat with the
/// exact same `(offset, len)`, so a plain map keyed on the pair hits
/// every time. Large data reads bypass the cache.
pub struct CachingReadAt<'a, R: IsoReadAt + Send> {
inner: &'a mut R,
cache: HashMap<(u64, u32), Vec<u8>>,
}
/// Don't cache reads bigger than this (file payloads, bulk scans).
const CACHE_MAX_READ: u32 = 256 * 1024;
/// Bound the cache so a pathological image can't grow it unbounded.
const CACHE_MAX_ENTRIES: usize = 256;
impl<'a, R: IsoReadAt + Send> CachingReadAt<'a, R> {
pub fn new(inner: &'a mut R) -> Self {
Self {
inner,
cache: HashMap::new(),
}
}
}
impl<R: IsoReadAt + Send> IsoReadAt for CachingReadAt<'_, R> {
async fn read_at(&mut self, offset: u64, len: u32) -> std::io::Result<Vec<u8>> {
let key = (offset, len);
if let Some(hit) = self.cache.get(&key) {
return Ok(hit.clone());
}
let buf = self.inner.read_at(offset, len).await?;
if len <= CACHE_MAX_READ && self.cache.len() < CACHE_MAX_ENTRIES {
self.cache.insert(key, buf.clone());
}
Ok(buf)
}
}
/// Look up `in_iso_path` (leading slash optional, case-insensitive) in
/// the image behind `r`. Returns `None` on any parsing or IO failure —
/// "not found" and "couldn't read" are the same answer to a prober.
///
/// v0.7.5: tries the primary ISO9660 namespace first, then falls back
/// to the **Joliet** supplementary namespace. Windows-oriented mastering
/// tools (common for appliance ISOs) often write a minimal or mangled
/// primary tree and keep the real filenames only in Joliet — without the
/// fallback those images probed as "no installer files" and their in-ISO
/// kernel fetches 404'd.
pub async fn lookup<R: IsoReadAt + Send>(r: &mut R, in_iso_path: &str) -> Option<FileLocation> {
let components: Vec<&str> = in_iso_path
.trim_start_matches('/')
.split('/')
.filter(|c| !c.is_empty())
.collect();
if components.is_empty() {
return None;
}
if let Some(root) = find_root(r, false).await {
if let Some(loc) = walk_namespace(r, root, &components, false).await {
return Some(loc);
}
}
if let Some(root) = find_root(r, true).await {
if let Some(loc) = walk_namespace(r, root, &components, true).await {
return Some(loc);
}
}
None
}
/// Find the namespace root: the Primary Volume Descriptor (`joliet =
/// false`) or the Joliet Supplementary Volume Descriptor (`joliet =
/// true`, identified by its UCS-2 escape sequence). Scans the whole
/// descriptor area rather than assuming fixed sectors, skipping any
/// non-`CD001` sector — sloppy mastering tools leave gaps. Returns the
/// root directory's `(lba, len)`.
async fn find_root<R: IsoReadAt + Send>(r: &mut R, joliet: bool) -> Option<(u64, u64)> {
let vd = find_descriptor(r, joliet).await?;
// Root directory record at descriptor offset 156, 34 bytes.
parse_dir_record_ext(&vd[156..156 + 34])
}
/// Scan the Volume Descriptor Set (LBA 16..32) for the wanted
/// descriptor: PVD (type 0x01) or Joliet SVD (type 0x02 carrying a
/// UCS-2 level 1/2/3 escape sequence at offset 88). Tolerant of
/// non-`CD001` filler sectors; stops at the Set Terminator.
pub(crate) async fn find_descriptor<R: IsoReadAt + Send>(
r: &mut R,
joliet: bool,
) -> Option<Vec<u8>> {
for lba in 16u64..32 {
let Ok(vd) = r.read_at(lba * SECTOR, 2048).await else {
return None;
};
if &vd[1..6] != b"CD001" {
continue;
}
match vd[0] {
0x01 if !joliet => return Some(vd),
0x02 if joliet && has_joliet_escape(&vd) => return Some(vd),
0xFF => return None,
_ => {}
}
}
None
}
/// Joliet SVDs declare a UCS-2 escape sequence at offset 88: `%/@`,
/// `%/C`, or `%/E` (levels 13).
fn has_joliet_escape(vd: &[u8]) -> bool {
matches!(vd.get(88..91), Some([0x25, 0x2F, 0x40 | 0x43 | 0x45]))
}
/// Walk path components down one namespace's directory tree. The
/// original walk was tail-recursive; iterate instead so the future
/// stays a plain (non-boxed) state machine.
async fn walk_namespace<R: IsoReadAt + Send>(
r: &mut R,
root: (u64, u64),
components: &[&str],
joliet: bool,
) -> Option<FileLocation> {
let (mut lba, mut len) = root;
for (idx, comp) in components.iter().enumerate() {
if len == 0 || len > MAX_DIR_BYTES {
return None;
}
let dir = r.read_at(lba * SECTOR, len as u32).await.ok()?;
let hit = scan_dir(&dir, comp, joliet)?;
let last = idx + 1 == components.len();
match (last, hit.is_dir) {
(true, false) => {
return Some(FileLocation {
offset: hit.lba * SECTOR,
length: hit.len,
})
}
(false, true) => {
lba = hit.lba;
len = hit.len;
}
_ => return None,
}
}
None
}
/// Convenience probe: does `in_iso_path` exist as a file?
pub async fn exists<R: IsoReadAt + Send>(r: &mut R, in_iso_path: &str) -> bool {
lookup(r, in_iso_path).await.is_some()
}
/// Synchronous wrapper for local files — the shape the HTTP handler's
/// `spawn_blocking` call site wants. `block_on` is safe here because
/// `FileReadAt`'s reads never actually await (they complete inline), so
/// the executor never parks.
#[must_use]
pub fn lookup_local(iso_path: &Path, in_iso_path: &str) -> Option<FileLocation> {
let f = std::fs::File::open(iso_path).ok()?;
futures::executor::block_on(lookup(&mut FileReadAt::new(f), in_iso_path))
}
struct DirHit {
lba: u64,
len: u64,
is_dir: bool,
}
/// Scan one directory extent for an identifier. Pure function over the
/// buffered extent — all protocol/IO concerns live in the caller.
/// `joliet` switches the identifier decoding (UCS-2 big-endian vs
/// d-characters); the record layout is otherwise identical.
fn scan_dir(dir: &[u8], target: &str, joliet: bool) -> Option<DirHit> {
let mut i = 0;
while i < dir.len() {
let len = dir[i] as usize;
if len == 0 {
// Records never span sectors; a zero length byte means the
// rest of this sector is padding. Hop to the next one.
let next = (i / SECTOR as usize + 1) * SECTOR as usize;
if next <= i {
break;
}
i = next;
continue;
}
if i + len > dir.len() {
break;
}
let rec = &dir[i..i + len];
let name = dir_record_name(rec, joliet);
let is_dir = (rec.get(25).copied().unwrap_or(0) & 0x02) != 0;
// Skip "." (0x00) and ".." (0x01) pseudo-entries.
let is_pseudo =
rec.get(32).copied() == Some(1) && matches!(rec.get(33).copied(), Some(0x00 | 0x01));
if !is_pseudo && name.eq_ignore_ascii_case(target) {
let (lba, dlen) = parse_dir_record_ext(rec)?;
return Some(DirHit {
lba,
len: dlen,
is_dir,
});
}
i += len;
}
None
}
/// Extract (extent LBA, data length in bytes) from a directory record.
/// Layout per ISO9660: bytes 2..10 extent LBA (LE+BE duplicate), 10..18
/// data length (LE+BE duplicate). We trust the little-endian copy.
fn parse_dir_record_ext(rec: &[u8]) -> Option<(u64, u64)> {
if rec.len() < 34 {
return None;
}
let lba = u64::from(u32::from_le_bytes(rec[2..6].try_into().ok()?));
let len = u64::from(u32::from_le_bytes(rec[10..14].try_into().ok()?));
Some((lba, len))
}
/// Extract the identifier from a directory record, normalizing ISO9660
/// quirks: the `;N` version suffix and the trailing dot that strict
/// mastering appends to extension-less names (`VMLINUZ.;1`). Without the
/// dot strip, level-1 images' kernels never matched `/casper/vmlinuz`.
/// Joliet identifiers are UCS-2 big-endian; decode then normalize the
/// same way (the `;1` suffix is two UCS-2 characters there).
fn dir_record_name(rec: &[u8], joliet: bool) -> String {
if joliet {
let name_len = *rec.get(32).unwrap_or(&0) as usize;
if name_len < 2 || rec.len() < 33 + name_len {
return String::new();
}
let raw = &rec[33..33 + name_len];
let units: Vec<u16> = raw
.chunks_exact(2)
.map(|p| u16::from_be_bytes([p[0], p[1]]))
.collect();
let s = String::from_utf16_lossy(&units);
let s = s.rfind(';').map_or_else(|| s.as_str(), |i| &s[..i]);
return s.strip_suffix('.').unwrap_or(s).to_string();
}
primary_record_name(rec)
}
fn primary_record_name(rec: &[u8]) -> String {
let name_len = *rec.get(32).unwrap_or(&0) as usize;
if name_len == 0 || rec.len() < 33 + name_len {
return String::new();
}
let raw = &rec[33..33 + name_len];
let s = String::from_utf8_lossy(raw);
let s = s.rfind(';').map_or_else(|| s.as_ref(), |i| &s[..i]);
s.strip_suffix('.').unwrap_or(s).to_string()
}
// ── test support ─────────────────────────────────────────────────────
//
// A tiny ISO9660 image builder used by this module's tests, the
// introspection tests, and (behind the `test-image` feature) other
// crates' integration tests. Lays out: PVD @ LBA 16, optional El Torito
// boot record @ 17, set terminator @ 18, directories from LBA 20, file
// data after. Only what `lookup`/introspection read is populated.
#[cfg(any(test, feature = "test-image"))]
#[doc(hidden)]
pub mod testiso {
use super::SECTOR;
use std::collections::BTreeMap;
#[derive(Default)]
struct Node {
children: BTreeMap<String, Node>,
content: Option<Vec<u8>>,
}
pub struct TestIsoBuilder {
root: Node,
volume_label: String,
el_torito: bool,
joliet_only: bool,
}
impl TestIsoBuilder {
pub fn new(volume_label: &str) -> Self {
Self {
root: Node::default(),
volume_label: volume_label.to_string(),
el_torito: false,
joliet_only: false,
}
}
#[must_use]
pub fn el_torito(mut self, on: bool) -> Self {
self.el_torito = on;
self
}
/// Model the Windows-mastering worst case: the primary ISO9660
/// tree is empty (just `.`/`..` in the root) and every real name
/// lives only in the Joliet supplementary namespace. Exercises
/// the v0.7.5 Joliet fallback end to end.
#[must_use]
pub fn joliet_only(mut self, on: bool) -> Self {
self.joliet_only = on;
self
}
/// Add a file at `path` (e.g. "/casper/vmlinuz") with `content`.
#[must_use]
pub fn file(mut self, path: &str, content: &[u8]) -> Self {
let mut node = &mut self.root;
let comps: Vec<&str> = path
.trim_start_matches('/')
.split('/')
.filter(|c| !c.is_empty())
.collect();
for (i, c) in comps.iter().enumerate() {
node = node.children.entry((*c).to_string()).or_default();
if i + 1 == comps.len() {
node.content = Some(content.to_vec());
}
}
self
}
pub fn build(self) -> Vec<u8> {
// Pass 1: allocate extents. Primary directories first (1
// sector each), then an optional parallel set of Joliet
// directory extents, then file contents (shared by both
// namespaces — only the directory trees differ).
let mut next_lba: u64 = 20;
let mut dirs: Vec<(*const Node, u64)> = Vec::new();
fn alloc_dirs(n: &Node, next: &mut u64, out: &mut Vec<(*const Node, u64)>) {
out.push((std::ptr::from_ref(n), *next));
*next += 1;
for child in n.children.values() {
if child.content.is_none() {
alloc_dirs(child, next, out);
}
}
}
alloc_dirs(&self.root, &mut next_lba, &mut dirs);
let lba_of = |n: &Node| -> u64 {
dirs.iter()
.find(|(p, _)| std::ptr::eq(*p, n))
.map(|(_, l)| *l)
.expect("dir allocated")
};
let mut jdirs: Vec<(*const Node, u64)> = Vec::new();
if self.joliet_only {
alloc_dirs(&self.root, &mut next_lba, &mut jdirs);
}
let jlba_of = |n: &Node| -> u64 {
jdirs
.iter()
.find(|(p, _)| std::ptr::eq(*p, n))
.map(|(_, l)| *l)
.expect("joliet dir allocated")
};
let mut file_lbas: Vec<(*const Node, u64, usize)> = Vec::new();
fn alloc_files(n: &Node, next: &mut u64, out: &mut Vec<(*const Node, u64, usize)>) {
for child in n.children.values() {
if let Some(c) = &child.content {
out.push((std::ptr::from_ref(child), *next, c.len()));
*next += c.len().div_ceil(SECTOR as usize).max(1) as u64;
} else {
alloc_files(child, next, out);
}
}
}
alloc_files(&self.root, &mut next_lba, &mut file_lbas);
let file_lba_of = |n: &Node| -> u64 {
file_lbas
.iter()
.find(|(p, _, _)| std::ptr::eq(*p, n))
.map(|(_, l, _)| *l)
.expect("file allocated")
};
let total = next_lba as usize * SECTOR as usize;
let mut img = vec![0u8; total];
// Directory record encoder.
fn record(name_bytes: &[u8], lba: u64, len: u64, is_dir: bool) -> Vec<u8> {
let mut rec_len = 33 + name_bytes.len();
if rec_len % 2 == 1 {
rec_len += 1; // pad to even
}
let rec_len = rec_len.max(34);
let mut r = vec![0u8; rec_len];
r[0] = rec_len as u8;
r[2..6].copy_from_slice(&(lba as u32).to_le_bytes());
r[6..10].copy_from_slice(&(lba as u32).to_be_bytes());
r[10..14].copy_from_slice(&(len as u32).to_le_bytes());
r[14..18].copy_from_slice(&(len as u32).to_be_bytes());
if is_dir {
r[25] = 0x02;
}
r[32] = name_bytes.len() as u8;
r[33..33 + name_bytes.len()].copy_from_slice(name_bytes);
r
}
// Pass 2: write each directory extent. `joliet` switches the
// identifier encoding; `skip_children` writes a bare ./..
// directory (the mangled-primary worst case).
#[allow(clippy::too_many_arguments)]
fn write_dir(
img: &mut [u8],
n: &Node,
self_lba: u64,
parent_lba: u64,
dir_lba_of: &dyn Fn(&Node) -> u64,
file_lba_of: &dyn Fn(&Node) -> u64,
joliet: bool,
skip_children: bool,
) {
let base = self_lba as usize * SECTOR as usize;
let mut off = 0usize;
let mut put = |rec: Vec<u8>, off: &mut usize| {
img[base + *off..base + *off + rec.len()].copy_from_slice(&rec);
*off += rec.len();
};
put(record(&[0x00], self_lba, SECTOR, true), &mut off);
put(record(&[0x01], parent_lba, SECTOR, true), &mut off);
if skip_children {
return;
}
let encode = |name: &str, file: bool| -> Vec<u8> {
if joliet {
// Joliet preserves case; files still carry `;1`.
let s = if file {
format!("{name};1")
} else {
name.to_string()
};
s.encode_utf16().flat_map(u16::to_be_bytes).collect()
} else if file {
// Primary gets the ISO9660 uppercase `;1` treatment
// so case-insensitive + version-strip matching is
// what the tests actually exercise.
format!("{};1", name.to_ascii_uppercase()).into_bytes()
} else {
name.to_ascii_uppercase().into_bytes()
}
};
for (name, child) in &n.children {
if let Some(c) = &child.content {
put(
record(
&encode(name, true),
file_lba_of(child),
c.len() as u64,
false,
),
&mut off,
);
} else {
put(
record(&encode(name, false), dir_lba_of(child), SECTOR, true),
&mut off,
);
}
}
for child in n.children.values() {
if child.content.is_none() {
write_dir(
img,
child,
dir_lba_of(child),
self_lba,
dir_lba_of,
file_lba_of,
joliet,
false,
);
} else if let Some(c) = &child.content {
let b = file_lba_of(child) as usize * SECTOR as usize;
img[b..b + c.len()].copy_from_slice(c);
}
}
}
let root_lba = lba_of(&self.root);
write_dir(
&mut img,
&self.root,
root_lba,
root_lba,
&lba_of,
&file_lba_of,
false,
self.joliet_only,
);
let jroot_lba = if self.joliet_only {
let jroot = jlba_of(&self.root);
write_dir(
&mut img,
&self.root,
jroot,
jroot,
&jlba_of,
&file_lba_of,
true,
false,
);
Some(jroot)
} else {
None
};
// PVD @ 16.
let pvd = 16 * SECTOR as usize;
img[pvd] = 0x01;
img[pvd + 1..pvd + 6].copy_from_slice(b"CD001");
let label = self.volume_label.as_bytes();
let label_field = &mut img[pvd + 40..pvd + 72];
label_field.fill(b' ');
label_field[..label.len().min(32)].copy_from_slice(&label[..label.len().min(32)]);
let root_rec = record(&[0x00], root_lba, SECTOR, true);
img[pvd + 156..pvd + 156 + 34].copy_from_slice(&root_rec[..34]);
// Optional El Torito boot record @ 17, then the optional
// Joliet SVD, then the set terminator.
let mut vd = 17 * SECTOR as usize;
if self.el_torito {
img[vd] = 0x00;
img[vd + 1..vd + 6].copy_from_slice(b"CD001");
let id = b"EL TORITO SPECIFICATION";
img[vd + 7..vd + 7 + id.len()].copy_from_slice(id);
vd += SECTOR as usize;
}
if let Some(jroot) = jroot_lba {
img[vd] = 0x02;
img[vd + 1..vd + 6].copy_from_slice(b"CD001");
// Joliet level-3 UCS-2 escape sequence.
img[vd + 88..vd + 91].copy_from_slice(&[0x25, 0x2F, 0x45]);
let jroot_rec = record(&[0x00], jroot, SECTOR, true);
img[vd + 156..vd + 156 + 34].copy_from_slice(&jroot_rec[..34]);
vd += SECTOR as usize;
}
img[vd] = 0xFF;
img[vd + 1..vd + 6].copy_from_slice(b"CD001");
img
}
}
/// In-memory `IsoReadAt` over a built test image.
pub struct MemReadAt(pub Vec<u8>);
impl super::IsoReadAt for MemReadAt {
async fn read_at(&mut self, offset: u64, len: u32) -> std::io::Result<Vec<u8>> {
let start = usize::try_from(offset).unwrap_or(usize::MAX);
let end = start.saturating_add(len as usize);
if end > self.0.len() {
return Err(std::io::Error::new(
std::io::ErrorKind::UnexpectedEof,
"read past end of test image",
));
}
Ok(self.0[start..end].to_vec())
}
}
}
#[cfg(test)]
mod tests {
use super::testiso::{MemReadAt, TestIsoBuilder};
use super::*;
fn block_on<T>(f: impl Future<Output = T>) -> T {
futures::executor::block_on(f)
}
#[test]
fn lookup_finds_nested_file_case_insensitively() {
let img = TestIsoBuilder::new("UBUNTU 24.04")
.file("/casper/vmlinuz", b"KERNELDATA")
.file("/casper/initrd", b"INITRDDATA")
.build();
let mut r = MemReadAt(img);
let loc = block_on(lookup(&mut r, "/CASPER/VMLINUZ")).expect("found");
assert_eq!(loc.length, 10);
let bytes = block_on(r.read_at(loc.offset, 10)).unwrap();
assert_eq!(&bytes, b"KERNELDATA");
// Missing file and missing dir both miss cleanly.
assert!(block_on(lookup(&mut r, "/casper/missing")).is_none());
assert!(block_on(lookup(&mut r, "/nodir/vmlinuz")).is_none());
// A directory path that resolves to a directory is not a file hit.
assert!(block_on(lookup(&mut r, "/casper")).is_none());
}
#[test]
fn strict_mastered_extensionless_names_match() {
// Strict level-1 mastering stores "VMLINUZ" as "VMLINUZ.;1" — the
// trailing dot must be normalized away or kernels never match.
let img = TestIsoBuilder::new("STRICT")
.file("/boot/vmlinuz.", b"K") // builder stores "VMLINUZ.;1"
.build();
let mut r = MemReadAt(img);
assert!(
block_on(lookup(&mut r, "/boot/vmlinuz")).is_some(),
"trailing-dot ISO9660 name must match the dotless path"
);
}
#[test]
fn lookup_three_levels_deep() {
let img = TestIsoBuilder::new("DEEP")
.file("/images/pxeboot/vmlinuz", b"ANACONDA")
.build();
let mut r = MemReadAt(img);
let loc = block_on(lookup(&mut r, "images/pxeboot/vmlinuz")).expect("no leading slash ok");
assert_eq!(loc.length, 8);
}
#[test]
fn caching_reader_dedupes_repeated_directory_reads() {
struct Counting<'a> {
inner: &'a mut MemReadAt,
calls: usize,
}
impl IsoReadAt for Counting<'_> {
async fn read_at(&mut self, offset: u64, len: u32) -> std::io::Result<Vec<u8>> {
self.calls += 1;
self.inner.read_at(offset, len).await
}
}
let img = TestIsoBuilder::new("CACHE")
.file("/a/one", b"1")
.file("/a/two", b"2")
.build();
let mut mem = MemReadAt(img);
let mut counting = Counting {
inner: &mut mem,
calls: 0,
};
let mut cr = CachingReadAt::new(&mut counting);
assert!(block_on(exists(&mut cr, "/a/one")));
assert!(block_on(exists(&mut cr, "/a/two")));
assert!(!block_on(exists(&mut cr, "/a/three")));
drop(cr);
// 3 probes × (PVD + root dir + subdir) collapse to the 3 distinct
// extents, plus one: the "/a/three" miss falls back to the Joliet
// namespace search (v0.7.5), which reads the terminator sector
// once before concluding there is no SVD.
assert_eq!(counting.calls, 4, "all repeat reads must hit the cache");
}
#[test]
fn joliet_fallback_finds_names_missing_from_primary() {
// Windows-mastering worst case: primary tree is bare (./.. only),
// real names live only in the Joliet SVD. The lookup must fall
// back and still resolve nested paths case-insensitively.
let img = TestIsoBuilder::new("APPLIANCE")
.joliet_only(true)
.file("/images/pxeboot/vmlinuz", b"JKERNEL")
.file("/sources/boot.wim", b"JWIM")
.build();
let mut r = MemReadAt(img);
let loc = block_on(lookup(&mut r, "/images/pxeboot/vmlinuz")).expect("joliet fallback");
let bytes = block_on(r.read_at(loc.offset, loc.length as u32)).unwrap();
assert_eq!(&bytes, b"JKERNEL");
assert!(block_on(lookup(&mut r, "/SOURCES/BOOT.WIM")).is_some());
assert!(block_on(lookup(&mut r, "/images/pxeboot/missing")).is_none());
}
#[test]
fn lookup_local_reads_a_real_file() {
let dir = tempfile::tempdir().unwrap();
let p = dir.path().join("t.iso");
let img = TestIsoBuilder::new("LOCAL")
.file("/sources/boot.wim", b"WIMWIM")
.build();
std::fs::write(&p, &img).unwrap();
let loc = lookup_local(&p, "/sources/boot.wim").expect("found");
assert_eq!(loc.length, 6);
assert!(lookup_local(&p, "/sources/none").is_none());
}
}
+8
View File
@@ -18,8 +18,15 @@
pub mod entry; pub mod entry;
pub mod introspect; pub mod introspect;
// v0.7.4: read-only ISO9660 walker generic over any random-access byte
// source (local file, NFS READ3, SFTP seek-read). Powers both in-ISO
// HTTP serving and the probe-based introspection.
pub mod iso_fs;
pub mod nfs_share; pub mod nfs_share;
pub mod pxe_logo; pub mod pxe_logo;
// v0.7.4: persisted cache of remote-share introspection results so a
// container restart doesn't re-probe an unchanged 40-ISO library.
pub mod remote_cache;
pub mod sftp_share; pub mod sftp_share;
pub mod smb; pub mod smb;
pub mod smb_share; pub mod smb_share;
@@ -29,6 +36,7 @@ pub mod windows;
pub use entry::{BootEntry, BootKind, KernelArgs}; pub use entry::{BootEntry, BootKind, KernelArgs};
pub use introspect::{DistroFamily, IntrospectionReport}; pub use introspect::{DistroFamily, IntrospectionReport};
pub use iso_fs::FileLocation;
// v0.4.65: kernel-mount NFS is gone. SMB shares via Samba's userspace // v0.4.65: kernel-mount NFS is gone. SMB shares via Samba's userspace
// `smbclient` CLI replaced it — works in any container (no // `smbclient` CLI replaced it — works in any container (no
// CAP_SYS_ADMIN, no host kernel modules), matching how Bootimus and // CAP_SYS_ADMIN, no host kernel modules), matching how Bootimus and
+204 -5
View File
@@ -57,7 +57,9 @@
//! UI to ask for. (If a future server needs Kerberos or non-default //! UI to ask for. (If a future server needs Kerberos or non-default
//! uid mapping we can add those, but for ISO read access nobody does.) //! uid mapping we can add those, but for ISO read access nobody does.)
use crate::introspect::IntrospectionReport; use crate::introspect::{introspect_reader, provisional_report};
use crate::iso_fs::{self, FileLocation, IsoReadAt};
use crate::remote_cache::RemoteIntrospectCache;
use crate::store::{generate_boot_entries_for, slugify_str, IsoSource, IsoStore}; use crate::store::{generate_boot_entries_for, slugify_str, IsoSource, IsoStore};
use bytes::Bytes; use bytes::Bytes;
use nfs3_client::tokio::TokioConnector; use nfs3_client::tokio::TokioConnector;
@@ -100,6 +102,18 @@ const READ_CHUNK_BYTES: u32 = 64 * 1024;
/// client park gigabytes of decoded ISO in RAM. /// client park gigabytes of decoded ISO in RAM.
const STREAM_BUFFER_DEPTH: usize = 16; const STREAM_BUFFER_DEPTH: usize = 16;
/// v0.7.4: per-ISO budget for a background introspection probe. A probe
/// is one connection plus a few dozen KiB-sized reads — sub-second on a
/// LAN — so anything past this is a wedged server, not a slow one.
const INTROSPECT_TIMEOUT: Duration = Duration::from_secs(30);
/// One queued background-introspection unit (v0.7.4).
struct ProbeJob {
iso_id: String,
filename: String,
size: u64,
}
/// One configured NFS share. The id is derived from server+export so /// One configured NFS share. The id is derived from server+export so
/// re-adding the same coordinates is idempotent. /// re-adding the same coordinates is idempotent.
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
@@ -177,6 +191,9 @@ pub struct NfsShareManager {
/// opens its own NFS connection so concurrency isn't a hard /// opens its own NFS connection so concurrency isn't a hard
/// requirement, but serializing keeps log output predictable. /// requirement, but serializing keeps log output predictable.
op_lock: Arc<tokio::sync::Mutex<()>>, op_lock: Arc<tokio::sync::Mutex<()>>,
/// v0.7.4: persisted introspection results keyed `share/path@size`,
/// so a restart re-probes only new or replaced ISOs.
introspect_cache: RemoteIntrospectCache,
} }
impl NfsShareManager { impl NfsShareManager {
@@ -190,6 +207,7 @@ impl NfsShareManager {
inner: Arc::new(Mutex::new(Inner::default())), inner: Arc::new(Mutex::new(Inner::default())),
iso_store, iso_store,
op_lock: Arc::new(tokio::sync::Mutex::new(())), op_lock: Arc::new(tokio::sync::Mutex::new(())),
introspect_cache: RemoteIntrospectCache::open(work_dir, "nfs_introspect_cache.json"),
} }
} }
@@ -387,12 +405,26 @@ impl NfsShareManager {
}; };
let mut count = 0u32; let mut count = 0u32;
let mut to_probe: Vec<ProbeJob> = Vec::new();
for entry in listing { for entry in listing {
let iso_id = format!("nfs-{}-{}", share.id, slugify_str(&entry.filename)); let iso_id = format!("nfs-{}-{}", share.id, slugify_str(&entry.filename));
// Same approach as SMB: no real introspection over the // v0.7.4: real introspection over the share — NFSv3 READ3
// network in v0.4.67. The boot-entry generator falls back // takes an offset, so the ISO9660 probes work remotely. A
// to filename-based sanboot detection. // cache hit registers the full report immediately; a miss
let report = IntrospectionReport::default(); // registers a provisional filename-based report (so the scan
// returns fast) and queues a background probe that upgrades
// the entry in place.
let cached = self
.introspect_cache
.get(&share.id, &entry.filename, entry.size);
let report = cached.unwrap_or_else(|| {
to_probe.push(ProbeJob {
iso_id: iso_id.clone(),
filename: entry.filename.clone(),
size: entry.size,
});
provisional_report(&entry.filename)
});
let boot_entries = generate_boot_entries_for(&iso_id, &entry.filename, &report); let boot_entries = generate_boot_entries_for(&iso_id, &entry.filename, &report);
let source = IsoSource::Nfs { let source = IsoSource::Nfs {
share_id: share.id.clone(), share_id: share.id.clone(),
@@ -413,11 +445,88 @@ impl NfsShareManager {
target: "openpxe::nfs", target: "openpxe::nfs",
id = %id, server = %share.server, export = %share.export, id = %id, server = %share.server, export = %share.export,
iso_count = count, iso_count = count,
pending_introspection = to_probe.len(),
"NFS share scanned" "NFS share scanned"
); );
if !to_probe.is_empty() {
self.spawn_introspection_pass(&share, to_probe);
}
Ok(count) Ok(count)
} }
/// v0.7.4: probe each queued ISO over its own NFS connection and swap
/// the full introspection into the store as results land. Runs
/// detached so neither startup nor the share-add API call waits on a
/// 40-ISO library; per-ISO failures (or a share removed mid-pass)
/// leave the provisional entry in place, which still sanboots.
fn spawn_introspection_pass(&self, share: &NfsShare, work: Vec<ProbeJob>) {
let store = self.iso_store.clone();
let cache = self.introspect_cache.clone();
let share_id = share.id.clone();
let server = share.server.clone();
let export = share.export.clone();
let port = share.port;
let queued = work.len();
tokio::spawn(async move {
let mut upgraded = 0usize;
for job in work {
let probe = async {
let mut reader = NfsReadAt::open(&server, &export, port, &job.filename).await?;
let report =
introspect_reader(&mut reader, job.size, &job.filename, false).await;
reader.finish().await;
Ok::<_, NfsClientError>(report)
};
match tokio::time::timeout(INTROSPECT_TIMEOUT, probe).await {
Ok(Ok(report)) => {
cache.put(&share_id, &job.filename, job.size, report.clone());
if store.update_external_introspection(&job.iso_id, report) {
upgraded += 1;
}
}
Ok(Err(e)) => tracing::warn!(
target: "openpxe::nfs",
share = %share_id, iso = %job.filename,
"introspection failed: {e}"
),
Err(_) => tracing::warn!(
target: "openpxe::nfs",
share = %share_id, iso = %job.filename,
"introspection timed out after {}s", INTROSPECT_TIMEOUT.as_secs()
),
}
}
tracing::info!(
target: "openpxe::nfs",
share = %share_id, queued, upgraded,
"remote introspection pass complete"
);
});
}
/// v0.7.4: locate `in_iso_path` inside a share-hosted ISO. Returns the
/// byte range so the HTTP layer can serve kernel/initrd files out of
/// remote ISOs with a follow-up ranged [`Self::stream_iso`].
pub async fn locate_in_iso(
&self,
share_id: &str,
filename: &str,
in_iso_path: &str,
) -> Result<Option<FileLocation>> {
let share = self
.get(share_id)
.ok_or_else(|| Error::Invalid(format!("no such NFS share '{share_id}'")))?;
if filename.contains('/') || filename.contains('\\') || filename.contains("..") {
return Err(Error::Invalid(format!("invalid filename '{filename}'")));
}
let mut reader = NfsReadAt::open(&share.server, &share.export, share.port, filename)
.await
.map_err(|e| Error::Invalid(format!("nfs open '{filename}': {e}")))?;
let loc = iso_fs::lookup(&mut reader, in_iso_path).await;
reader.finish().await;
Ok(loc)
}
fn update_status( fn update_status(
&self, &self,
id: &str, id: &str,
@@ -490,6 +599,96 @@ struct NfsListEntry {
size: u64, size: u64,
} }
/// The connection type [`build_connection`] yields.
type NfsConn = nfs3_client::Nfs3Connection<nfs3_client::tokio::TokioIo<tokio::net::TcpStream>>;
/// v0.7.4: random-access reader over one NFS connection + file handle —
/// the [`IsoReadAt`] impl that lets the ISO9660 walker and introspection
/// probes run against share-hosted images.
struct NfsReadAt {
conn: NfsConn,
fh: nfs_fh3,
}
impl NfsReadAt {
/// Connect, mount, and LOOKUP `filename` at the export root.
async fn open(
server: &str,
export: &str,
port: u16,
filename: &str,
) -> std::result::Result<Self, NfsClientError> {
let mut conn = build_connection(server, export, port).await?;
let root = conn.root_nfs_fh3();
let lookup = conn
.lookup(&LOOKUP3args {
what: diropargs3 {
dir: root,
name: filename3(Opaque::borrowed(filename.as_bytes())),
},
})
.await
.map_err(NfsClientError::Rpc)?;
let fh = match lookup {
Nfs3Result::Ok(o) => o.object,
Nfs3Result::Err((status, _)) => {
return Err(NfsClientError::Nfsstat(status_label(status)));
}
};
Ok(Self { conn, fh })
}
/// Best-effort unmount. Consumes the reader — it's done.
async fn finish(self) {
let _ = self.conn.unmount().await;
}
}
impl IsoReadAt for NfsReadAt {
async fn read_at(&mut self, offset: u64, len: u32) -> std::io::Result<Vec<u8>> {
let mut out: Vec<u8> = Vec::with_capacity(len as usize);
let mut off = offset;
// READ3 may legally return fewer bytes than asked (server cap);
// loop until the exact-read contract is satisfied or the file
// genuinely ends short.
while (out.len() as u32) < len {
let want = (len - out.len() as u32).min(READ_CHUNK_BYTES);
let res = self
.conn
.read(&READ3args {
file: self.fh.clone(),
offset: off,
count: want,
})
.await
.map_err(|e| std::io::Error::other(e.to_string()))?;
let ok = match res {
Nfs3Result::Ok(o) => o,
Nfs3Result::Err((status, _)) => {
return Err(std::io::Error::other(status_label(status)));
}
};
let data = ok.data.as_ref();
if data.is_empty() {
return Err(std::io::Error::new(
std::io::ErrorKind::UnexpectedEof,
"NFS read past end of file",
));
}
out.extend_from_slice(data);
off += data.len() as u64;
if ok.eof && (out.len() as u32) < len {
return Err(std::io::Error::new(
std::io::ErrorKind::UnexpectedEof,
"NFS read past end of file",
));
}
}
out.truncate(len as usize);
Ok(out)
}
}
/// Connect, READDIR the export root, look up each `*.iso` to get its /// Connect, READDIR the export root, look up each `*.iso` to get its
/// size + file handle. Returns a flat list. Errors are returned with /// size + file handle. Returns a flat list. Errors are returned with
/// a human-readable message; the caller decides how to surface them. /// a human-readable message; the caller decides how to surface them.
+142
View File
@@ -0,0 +1,142 @@
//! Persisted cache of remote-share introspection results.
//!
//! NFS/SFTP introspection costs a connection plus a few dozen small
//! reads per ISO. Shares are rescanned on every startup and share-add,
//! so without a cache a 40-ISO library would re-probe 40 ISOs on every
//! container restart. The cache keys on `share/path@size` — a replaced
//! file (new size) re-probes, an untouched one is free — and entries
//! only count as hits when their `introspect_rev` matches the current
//! logic, so an upgrade that changes detection re-probes everything
//! exactly once.
//!
//! One file per protocol (`nfs_introspect_cache.json`,
//! `sftp_introspect_cache.json`) so the two managers never contend over
//! one writer.
use crate::introspect::{IntrospectionReport, INTROSPECT_REV};
use parking_lot::Mutex;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::Arc;
/// Hard cap on cached entries; beyond it the cache resets rather than
/// growing unbounded (a cache wipe only costs one re-probe pass).
const MAX_ENTRIES: usize = 4096;
#[derive(Clone, Debug)]
pub struct RemoteIntrospectCache {
path: Arc<PathBuf>,
map: Arc<Mutex<HashMap<String, IntrospectionReport>>>,
}
impl RemoteIntrospectCache {
/// Open (or start empty) the cache at `<work_dir>/<file_name>`.
/// A corrupt or missing file is an empty cache, never an error.
#[must_use]
pub fn open(work_dir: &Path, file_name: &str) -> Self {
let path = work_dir.join(file_name);
let map = std::fs::read_to_string(&path)
.ok()
.and_then(|text| {
serde_json::from_str::<HashMap<String, IntrospectionReport>>(&text).ok()
})
.unwrap_or_default();
Self {
path: Arc::new(path),
map: Arc::new(Mutex::new(map)),
}
}
fn key(share_id: &str, relative_path: &str, size: u64) -> String {
format!("{share_id}/{relative_path}@{size}")
}
/// A hit requires the entry to have been produced by the *current*
/// introspection logic — stale-rev entries are misses, which is how
/// the cache self-invalidates across upgrades.
#[must_use]
pub fn get(
&self,
share_id: &str,
relative_path: &str,
size: u64,
) -> Option<IntrospectionReport> {
self.map
.lock()
.get(&Self::key(share_id, relative_path, size))
.filter(|r| r.introspect_rev == INTROSPECT_REV)
.cloned()
}
pub fn put(&self, share_id: &str, relative_path: &str, size: u64, report: IntrospectionReport) {
let snapshot = {
let mut g = self.map.lock();
if g.len() >= MAX_ENTRIES {
g.clear();
}
g.insert(Self::key(share_id, relative_path, size), report);
g.clone()
};
// Persist outside the lock; tmp+rename so a crash mid-write
// leaves the previous cache intact.
let path = self.path.as_path();
let tmp = path.with_extension("json.tmp");
let Ok(body) = serde_json::to_vec_pretty(&snapshot) else {
return;
};
if let Some(parent) = path.parent() {
let _ = std::fs::create_dir_all(parent);
}
if std::fs::write(&tmp, body).is_ok() {
let _ = std::fs::rename(&tmp, path);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::introspect::DistroFamily;
#[test]
fn round_trips_across_reopen_and_rev_gates() {
let dir = tempfile::tempdir().unwrap();
let cache = RemoteIntrospectCache::open(dir.path(), "t.json");
assert!(cache.get("s1", "a.iso", 100).is_none());
let fresh = IntrospectionReport {
family: DistroFamily::RhelFedora,
introspect_rev: INTROSPECT_REV,
el_torito: true,
..Default::default()
};
cache.put("s1", "a.iso", 100, fresh.clone());
assert_eq!(
cache.get("s1", "a.iso", 100).unwrap().family,
DistroFamily::RhelFedora
);
// Different size = different file = miss.
assert!(cache.get("s1", "a.iso", 101).is_none());
// Survives a reopen.
let cache2 = RemoteIntrospectCache::open(dir.path(), "t.json");
assert!(cache2.get("s1", "a.iso", 100).is_some());
// Stale-rev entries never hit.
let stale = IntrospectionReport {
family: DistroFamily::Arch,
introspect_rev: INTROSPECT_REV - 1,
..Default::default()
};
cache2.put("s1", "b.iso", 7, stale);
assert!(cache2.get("s1", "b.iso", 7).is_none());
}
#[test]
fn corrupt_cache_file_starts_empty() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("t.json"), b"{nope").unwrap();
let cache = RemoteIntrospectCache::open(dir.path(), "t.json");
assert!(cache.get("s", "x.iso", 1).is_none());
}
}
+150 -6
View File
@@ -56,7 +56,9 @@
//! hint}` error shape is shared so the storage tab renders all three //! hint}` error shape is shared so the storage tab renders all three
//! protocols through one code path. //! protocols through one code path.
use crate::introspect::IntrospectionReport; use crate::introspect::{introspect_reader, provisional_report};
use crate::iso_fs::{self, FileLocation, IsoReadAt};
use crate::remote_cache::RemoteIntrospectCache;
use crate::store::{generate_boot_entries_for, slugify_str, IsoSource, IsoStore}; use crate::store::{generate_boot_entries_for, slugify_str, IsoSource, IsoStore};
use bytes::Bytes; use bytes::Bytes;
use openpxe_core::{Error, Result}; use openpxe_core::{Error, Result};
@@ -96,6 +98,18 @@ const READ_CHUNK_BYTES: usize = 64 * 1024;
/// body stream. 16 * 64 KiB ≈ 1 MiB max buffer per stream. /// body stream. 16 * 64 KiB ≈ 1 MiB max buffer per stream.
const STREAM_BUFFER_DEPTH: usize = 16; const STREAM_BUFFER_DEPTH: usize = 16;
/// v0.7.4: per-ISO budget for a background introspection probe — one SSH
/// connection plus a few dozen KiB-sized reads. SSH handshakes cost more
/// than NFS mounts, but 30s still only trips on a wedged server.
const INTROSPECT_TIMEOUT: Duration = Duration::from_secs(30);
/// One queued background-introspection unit (v0.7.4).
struct ProbeJob {
iso_id: String,
filename: String,
size: u64,
}
/// Which credential the share authenticates with. The secret itself /// Which credential the share authenticates with. The secret itself
/// lives in the 0600 creds file, never here. /// lives in the 0600 creds file, never here.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
@@ -209,6 +223,9 @@ pub struct SftpShareManager {
/// Serializes scan operations on the same manager for predictable /// Serializes scan operations on the same manager for predictable
/// log output; each scan opens its own SSH connection. /// log output; each scan opens its own SSH connection.
op_lock: Arc<tokio::sync::Mutex<()>>, op_lock: Arc<tokio::sync::Mutex<()>>,
/// v0.7.4: persisted introspection results keyed `share/path@size`,
/// so a restart re-probes only new or replaced ISOs.
introspect_cache: RemoteIntrospectCache,
} }
impl SftpShareManager { impl SftpShareManager {
@@ -222,6 +239,7 @@ impl SftpShareManager {
inner: Arc::new(Mutex::new(Inner::default())), inner: Arc::new(Mutex::new(Inner::default())),
iso_store, iso_store,
op_lock: Arc::new(tokio::sync::Mutex::new(())), op_lock: Arc::new(tokio::sync::Mutex::new(())),
introspect_cache: RemoteIntrospectCache::open(work_dir, "sftp_introspect_cache.json"),
} }
} }
@@ -474,13 +492,25 @@ impl SftpShareManager {
}; };
let mut count = 0u32; let mut count = 0u32;
let mut to_probe: Vec<ProbeJob> = Vec::new();
for entry in listing { for entry in listing {
let iso_id = format!("sftp-{}-{}", share.id, slugify_str(&entry.filename)); let iso_id = format!("sftp-{}-{}", share.id, slugify_str(&entry.filename));
// Same as NFS/SMB: no over-the-network introspection yet, so // v0.7.4: real introspection over the share — SFTP file
// register `Unknown` and let the boot-entry generator fall // handles are seekable, so the ISO9660 probes work remotely.
// back to filename-based detection. SFTP *could* do bounded // Cache hit → full report now; miss → provisional filename-
// PVD reads (it has random access) a follow-up can add it. // based report (scan returns fast) + a queued background
let report = IntrospectionReport::default(); // probe that upgrades the entry in place.
let cached = self
.introspect_cache
.get(&share.id, &entry.filename, entry.size);
let report = cached.unwrap_or_else(|| {
to_probe.push(ProbeJob {
iso_id: iso_id.clone(),
filename: entry.filename.clone(),
size: entry.size,
});
provisional_report(&entry.filename)
});
let boot_entries = generate_boot_entries_for(&iso_id, &entry.filename, &report); let boot_entries = generate_boot_entries_for(&iso_id, &entry.filename, &report);
let source = IsoSource::Sftp { let source = IsoSource::Sftp {
share_id: share.id.clone(), share_id: share.id.clone(),
@@ -506,11 +536,88 @@ impl SftpShareManager {
target: "openpxe::sftp", target: "openpxe::sftp",
id = %id, server = %share.server, export = %share.export, id = %id, server = %share.server, export = %share.export,
iso_count = count, iso_count = count,
pending_introspection = to_probe.len(),
"SFTP share scanned" "SFTP share scanned"
); );
if !to_probe.is_empty() {
self.spawn_introspection_pass(&share, &creds, to_probe);
}
Ok(count) Ok(count)
} }
/// v0.7.4: probe each queued ISO over its own SSH connection and swap
/// the full introspection into the store as results land. Detached so
/// neither startup nor the share-add API call waits on a big library;
/// per-ISO failures leave the provisional entry, which still sanboots.
fn spawn_introspection_pass(&self, share: &SftpShare, creds: &SftpCreds, work: Vec<ProbeJob>) {
let store = self.iso_store.clone();
let cache = self.introspect_cache.clone();
let share_id = share.id.clone();
let params = ConnParams::from_share(share);
let creds = creds.clone();
let queued = work.len();
tokio::spawn(async move {
let mut upgraded = 0usize;
for job in work {
let probe = async {
let mut reader = SftpReadAt::open(&params, &creds, &job.filename).await?;
let report =
introspect_reader(&mut reader, job.size, &job.filename, false).await;
Ok::<_, SftpClientError>(report)
};
match tokio::time::timeout(INTROSPECT_TIMEOUT, probe).await {
Ok(Ok(report)) => {
cache.put(&share_id, &job.filename, job.size, report.clone());
if store.update_external_introspection(&job.iso_id, report) {
upgraded += 1;
}
}
Ok(Err(e)) => tracing::warn!(
target: "openpxe::sftp",
share = %share_id, iso = %job.filename,
"introspection failed: {e}"
),
Err(_) => tracing::warn!(
target: "openpxe::sftp",
share = %share_id, iso = %job.filename,
"introspection timed out after {}s", INTROSPECT_TIMEOUT.as_secs()
),
}
}
tracing::info!(
target: "openpxe::sftp",
share = %share_id, queued, upgraded,
"remote introspection pass complete"
);
});
}
/// v0.7.4: locate `in_iso_path` inside a share-hosted ISO. Returns the
/// byte range so the HTTP layer can serve kernel/initrd files out of
/// remote ISOs with a follow-up ranged [`Self::stream_iso`].
pub async fn locate_in_iso(
&self,
share_id: &str,
filename: &str,
in_iso_path: &str,
) -> Result<Option<FileLocation>> {
let share = self
.get(share_id)
.ok_or_else(|| Error::Invalid(format!("no such SFTP share '{share_id}'")))?;
if filename.contains('/') || filename.contains('\\') || filename.contains("..") {
return Err(Error::Invalid(format!("invalid filename '{filename}'")));
}
let creds = self
.read_creds(share_id)
.await
.map_err(|e| Error::Invalid(format!("could not read credentials: {e}")))?;
let params = ConnParams::from_share(&share);
let mut reader = SftpReadAt::open(&params, &creds, filename)
.await
.map_err(|e| Error::Invalid(format!("sftp open '{filename}': {e}")))?;
Ok(iso_fs::lookup(&mut reader, in_iso_path).await)
}
fn pin_fingerprint(&self, id: &str, fingerprint: String) { fn pin_fingerprint(&self, id: &str, fingerprint: String) {
if fingerprint.is_empty() { if fingerprint.is_empty() {
return; return;
@@ -652,6 +759,43 @@ struct SftpConn {
sftp: SftpSession, sftp: SftpSession,
} }
/// v0.7.4: random-access reader over one SSH connection + open file
/// handle — the [`IsoReadAt`] impl that lets the ISO9660 walker and
/// introspection probes run against share-hosted images. Holds the
/// `SftpConn` so the SSH session outlives every read.
struct SftpReadAt {
_conn: SftpConn,
file: russh_sftp::client::fs::File,
}
impl SftpReadAt {
async fn open(
p: &ConnParams,
creds: &SftpCreds,
filename: &str,
) -> std::result::Result<Self, SftpClientError> {
let (conn, _fp) = connect(p, creds).await?;
let full = format!("{}/{}", p.export.trim_end_matches('/'), filename);
let file = conn
.sftp
.open(full)
.await
.map_err(|e| SftpClientError::Sftp(e.to_string()))?;
Ok(Self { _conn: conn, file })
}
}
impl IsoReadAt for SftpReadAt {
async fn read_at(&mut self, offset: u64, len: u32) -> std::io::Result<Vec<u8>> {
self.file.seek(SeekFrom::Start(offset)).await?;
let mut buf = vec![0u8; len as usize];
// read_exact loops over the transport's short reads and fails
// with UnexpectedEof past end-of-file — exactly the contract.
self.file.read_exact(&mut buf).await?;
Ok(buf)
}
}
/// russh client handler implementing trust-on-first-use host-key /// russh client handler implementing trust-on-first-use host-key
/// verification. We never construct an `Err` from `check_server_key`; /// verification. We never construct an `Err` from `check_server_key`;
/// returning `Ok(false)` makes russh abort the handshake, and the /// returning `Ok(false)` makes russh abort the handshake, and the
+9 -10
View File
@@ -56,7 +56,7 @@
//! streaming. A follow-up release can add libsmbclient-based seek if //! streaming. A follow-up release can add libsmbclient-based seek if
//! a real workload needs it. //! a real workload needs it.
use crate::introspect::IntrospectionReport; use crate::introspect::provisional_report;
use crate::store::{generate_boot_entries_for, slugify_str, IsoSource, IsoStore}; use crate::store::{generate_boot_entries_for, slugify_str, IsoSource, IsoStore};
use openpxe_core::{Error, Result}; use openpxe_core::{Error, Result};
use parking_lot::Mutex; use parking_lot::Mutex;
@@ -455,15 +455,14 @@ impl SmbShareManager {
let mut count = 0u32; let mut count = 0u32;
for entry in listing { for entry in listing {
let iso_id = format!("smb-{}-{}", share.id, slugify_str(&entry.filename)); let iso_id = format!("smb-{}-{}", share.id, slugify_str(&entry.filename));
// SMB sources don't get a real introspection pass — that // SMB sources can't get the content-probe pass NFS/SFTP got
// would require seeking into the ISO9660 PVD over the // in v0.7.4 — the ISO9660 probes need seeks, and smbclient's
// network, and smbclient CLI doesn't seek. We register an // CLI streaming doesn't seek. The provisional filename-token
// `Unknown` family so the boot-entry generator falls back // report is as far as SMB detection goes: family for the UI
// to generic sanboot/wimboot detection from the filename // when the name says it ("rhel-9.0…", "Win11_…"), and
// and the operator gets *something* bootable. A follow-up // `introspect_rev = 0` so the entry generator keeps the
// release can do a bounded `smbclient get` of the first // optimistic sanboot entry.
// 64 KiB for real detection. let report = provisional_report(&entry.filename);
let report = IntrospectionReport::default();
let boot_entries = generate_boot_entries_for(&iso_id, &entry.filename, &report); let boot_entries = generate_boot_entries_for(&iso_id, &entry.filename, &report);
let source = IsoSource::Smb { let source = IsoSource::Smb {
share_id: share.id.clone(), share_id: share.id.clone(),
+62 -4
View File
@@ -300,7 +300,19 @@ impl IsoStore {
} }
let partial_path = self.iso_dir.join(format!("{id}.partial")); let partial_path = self.iso_dir.join(format!("{id}.partial"));
if partial_path.exists() { if partial_path.exists() {
return Err(Error::Invalid(format!("iso '{id}' is already uploading"))); // A leftover .partial is an upload abandoned mid-flight (browser
// refresh, tab close, dropped connection) — nothing reaps it
// otherwise, and the operator hits a bogus "already uploading"
// on retry. The chunked protocol can't resume it anyway (a
// fresh session restarts at offset 0), so reclaim it.
// ponytail: two tabs uploading the *same filename* at once would
// race here — last writer wins, and the truncating create below
// keeps that from corrupting a half-written file.
tracing::info!(
target: "openpxe::iso", %id,
"reclaiming abandoned .partial from a prior upload attempt"
);
tokio::fs::remove_file(&partial_path).await.ok();
} }
let file = tokio::fs::File::create(&partial_path).await?; let file = tokio::fs::File::create(&partial_path).await?;
Ok(UploadHandle { Ok(UploadHandle {
@@ -415,6 +427,28 @@ impl IsoStore {
self.inner.write().isos.insert(id, meta); self.inner.write().isos.insert(id, meta);
} }
/// v0.7.4: swap in a completed introspection for an external ISO and
/// regenerate its boot entries. Used by the NFS/SFTP managers'
/// background probe pass — the scan registers a provisional
/// (filename-only) report immediately so startup and share-add stay
/// fast, then this upgrades each entry as its probe finishes.
/// Operator-set fields (category, password) are preserved; returns
/// `false` when the id is gone (share removed or re-scanned away
/// mid-probe), which callers treat as a benign no-op.
pub fn update_external_introspection(
&self,
id: &str,
introspection: IntrospectionReport,
) -> bool {
let mut g = self.inner.write();
let Some(m) = g.isos.get_mut(id) else {
return false;
};
m.boot_entries = generate_boot_entries(&m.id, &m.filename, &introspection);
m.introspection = introspection;
true
}
/// Drop every entry that belongs to `share_id`. Used by the SMB /// Drop every entry that belongs to `share_id`. Used by the SMB
/// and NFS share managers when an operator removes a share, or /// and NFS share managers when an operator removes a share, or
/// before re-scanning to clean out stale entries. The same id /// before re-scanning to clean out stale entries. The same id
@@ -850,15 +884,39 @@ mod tests {
} }
#[tokio::test] #[tokio::test]
async fn begin_upload_rejects_existing_partial_file() { async fn begin_upload_reclaims_stale_partial_file() {
// v0.8.0: an abandoned .partial (browser refresh / crash / dropped
// connection) must not block a re-upload with a bogus "already
// uploading" — begin_upload reclaims it and starts fresh, since the
// chunked protocol can't resume a dead session anyway.
let dir = tempdir().unwrap(); let dir = tempdir().unwrap();
let store = IsoStore::new(dir.path().to_path_buf()); let store = IsoStore::new(dir.path().to_path_buf());
store.ensure_dirs().await.unwrap(); store.ensure_dirs().await.unwrap();
tokio::fs::write(dir.path().join("ubuntu.partial"), b"in-flight") let partial = dir.path().join("ubuntu.partial");
tokio::fs::write(&partial, b"in-flight").await.unwrap();
let handle = store
.begin_upload("ubuntu.iso")
.await
.expect("stale .partial is reclaimed, not rejected");
assert_eq!(handle.id, "ubuntu");
// Reclaimed: the leftover bytes are gone (fresh, empty file).
let meta = tokio::fs::metadata(&partial).await.unwrap();
assert_eq!(meta.len(), 0, "stale .partial must be truncated on reclaim");
}
#[tokio::test]
async fn begin_upload_still_rejects_completed_iso() {
// A finished upload (final .iso on disk) is a genuine duplicate, not
// an abandoned attempt — that case must still be refused.
let dir = tempdir().unwrap();
let store = IsoStore::new(dir.path().to_path_buf());
store.ensure_dirs().await.unwrap();
tokio::fs::write(dir.path().join("rocky.iso"), b"done")
.await .await
.unwrap(); .unwrap();
let r = store.begin_upload("ubuntu.iso").await; let r = store.begin_upload("rocky.iso").await;
assert!(matches!(r, Err(Error::Invalid(_)))); assert!(matches!(r, Err(Error::Invalid(_))));
} }
-2
View File
@@ -26,7 +26,5 @@ tracing.workspace = true
tracing-subscriber.workspace = true tracing-subscriber.workspace = true
anyhow.workspace = true anyhow.workspace = true
clap.workspace = true clap.workspace = true
serde.workspace = true
toml.workspace = true
bytes.workspace = true bytes.workspace = true
time.workspace = true time.workspace = true
+53 -1
View File
@@ -122,6 +122,7 @@ async fn main() -> anyhow::Result<()> {
let admin = openpxe_core::AdminStore::load_or_default(&config.paths.work_dir); let admin = openpxe_core::AdminStore::load_or_default(&config.paths.work_dir);
let sso = openpxe_core::SsoStore::load_or_default(&config.paths.work_dir); let sso = openpxe_core::SsoStore::load_or_default(&config.paths.work_dir);
let notify = openpxe_core::NotifyStore::load_or_default(&config.paths.work_dir); let notify = openpxe_core::NotifyStore::load_or_default(&config.paths.work_dir);
let api_key = openpxe_core::ApiKeyStore::load_or_init(&config.paths.work_dir);
let sessions = openpxe_http_api::auth::SessionStore::default(); let sessions = openpxe_http_api::auth::SessionStore::default();
let metrics = Metrics::new(); let metrics = Metrics::new();
@@ -184,6 +185,10 @@ async fn main() -> anyhow::Result<()> {
"network info" "network info"
); );
// v0.7.1: boot rules are shared between the HTTP layer (target rules,
// webhook, the editor API) and the DHCP proxy (driver-mode pins).
let boot_rules = openpxe_core::BootRulesStore::load_or_default(&config.paths.work_dir);
let state = AppState { let state = AppState {
iso_store: iso_store.clone(), iso_store: iso_store.clone(),
clients: clients.clone(), clients: clients.clone(),
@@ -191,12 +196,13 @@ async fn main() -> anyhow::Result<()> {
queue: queue.clone(), queue: queue.clone(),
hosts: hosts.clone(), hosts: hosts.clone(),
boot_log: boot_log.clone(), boot_log: boot_log.clone(),
boot_rules: openpxe_core::BootRulesStore::load_or_default(&config.paths.work_dir), boot_rules: boot_rules.clone(),
boot_tokens: openpxe_core::BootTokens::new(), boot_tokens: openpxe_core::BootTokens::new(),
branding: branding.clone(), branding: branding.clone(),
pxe_bg_cache: openpxe_http_api::state::PxeBgCache::default(), pxe_bg_cache: openpxe_http_api::state::PxeBgCache::default(),
admin: admin.clone(), admin: admin.clone(),
sessions: sessions.clone(), sessions: sessions.clone(),
api_key,
sso: sso.clone(), sso: sso.clone(),
saml: openpxe_http_api::saml_routes::SamlRuntime::default(), saml: openpxe_http_api::saml_routes::SamlRuntime::default(),
notify: notify.clone(), notify: notify.clone(),
@@ -211,6 +217,7 @@ async fn main() -> anyhow::Result<()> {
started_at: time::OffsetDateTime::now_utc(), started_at: time::OffsetDateTime::now_utc(),
public_base_url: public_base_url.clone(), public_base_url: public_base_url.clone(),
nic_name: net.nic_name, nic_name: net.nic_name,
nic_link: net.nic_link,
subnet_mask: net.subnet_mask, subnet_mask: net.subnet_mask,
gateway: net.gateway, gateway: net.gateway,
}; };
@@ -267,6 +274,10 @@ async fn main() -> anyhow::Result<()> {
public_base_url.clone(), public_base_url.clone(),
clients.clone(), clients.clone(),
metrics.clone(), metrics.clone(),
// v0.7.1: learned driver modes persist next to the other
// state files, so a machine walks the ladder once *ever*.
openpxe_dhcp_proxy::DriverEscalation::load_or_default(&config.paths.work_dir),
boot_rules.clone(),
); );
tokio::spawn(s.run()) tokio::spawn(s.run())
} }
@@ -440,6 +451,12 @@ struct NetworkInfo {
nic_name: String, nic_name: String,
subnet_mask: String, subnet_mask: String,
gateway: String, gateway: String,
/// v0.7.2: physical link summary for the Network tab — operstate,
/// negotiated speed/duplex, and the port's own MAC. Helps operators
/// in multi-NIC / trunked environments confirm *which* port the PXE
/// server actually answers on. Empty when sysfs isn't available
/// (non-Linux dev builds) or the NIC wasn't identified.
nic_link: String,
} }
/// Best-effort population of the Network tab's read-only fields. We shell /// Best-effort population of the Network tab's read-only fields. We shell
@@ -500,9 +517,44 @@ fn detect_network_info(our_ip: Ipv4Addr) -> NetworkInfo {
} }
} }
info.nic_link = detect_link_info(&info.nic_name);
info info
} }
/// v0.7.2: read the NIC's physical link details from sysfs. Every field
/// is optional — virtual NICs report no speed (`-1` or absent), and
/// non-Linux dev machines have no `/sys/class/net` at all — so the
/// result is whatever could be read, joined human-readably, or empty.
fn detect_link_info(nic: &str) -> String {
if nic.is_empty() {
return String::new();
}
let read = |file: &str| {
std::fs::read_to_string(format!("/sys/class/net/{nic}/{file}"))
.map(|s| s.trim().to_string())
.unwrap_or_default()
};
let mut parts: Vec<String> = Vec::new();
let state = read("operstate");
if !state.is_empty() {
parts.push(format!("link {state}"));
}
let speed = read("speed");
if !speed.is_empty() && speed != "-1" {
parts.push(format!("{speed} Mb/s"));
}
let duplex = read("duplex");
if !duplex.is_empty() && duplex != "unknown" {
parts.push(format!("{duplex} duplex"));
}
let mac = read("address");
if !mac.is_empty() {
parts.push(format!("port {mac}"));
}
parts.join(" · ")
}
fn prefix_to_dotted(prefix: u8) -> String { fn prefix_to_dotted(prefix: u8) -> String {
let prefix = prefix.min(32); let prefix = prefix.min(32);
let mask: u32 = if prefix == 0 { let mask: u32 = if prefix == 0 {
-2
View File
@@ -15,6 +15,4 @@ openpxe-ipxe-assets.workspace = true
tokio.workspace = true tokio.workspace = true
socket2.workspace = true socket2.workspace = true
tracing.workspace = true tracing.workspace = true
thiserror.workspace = true
anyhow.workspace = true anyhow.workspace = true
bytes.workspace = true
+25
View File
@@ -912,3 +912,28 @@ tr.unbootable td:first-child { border-left: 3px solid var(--warn); }
display: flex; justify-content: flex-end; gap: 10px; margin-top: 18px; display: flex; justify-content: flex-end; gap: 10px; margin-top: 18px;
} }
.modal-actions .submit { width: auto; padding: 8px 18px; } .modal-actions .submit { width: auto; padding: 8px 18px; }
/* v0.7.2: a label.field directly followed by the card's action button
stacked its own 14px bottom margin onto the button's 16px top margin
(30px total) visible on Queue "Launch for all waiting" and the
Network "Save". Collapse the doubled gap so every primary action sits
the same 16px below its form. */
.card .body > label.field:has(+ button) { margin-bottom: 0; }
/* v0.7.2: inline list filter above a table (Available images). The input
is wrapped in a label.field so it borrows the standard text-field chrome
and matches every other input in the app; this wrapper just insets it
from the card edges so it lines up with the header text above. */
.list-search { padding: 14px 16px; }
/* v0.7.4: pager footer under the Available-images table quiet status
text on the left, ghost Prev/Next on the right. */
.list-pager {
display: flex; align-items: center; gap: 8px;
padding: 12px 16px;
color: var(--fg-dim); font-size: 12px;
font-variant-numeric: tabular-nums;
}
.list-pager .spacer { flex: 1; }
.list-pager button { padding: 4px 12px; font-size: 12px; }
.list-pager button:disabled { opacity: 0.45; cursor: default; }
+368 -180
View File
@@ -178,12 +178,14 @@
if (iso.introspection.el_torito) { if (iso.introspection.el_torito) {
return { ok: true, warn: 'generic bootable ISO — boots via sanboot (emulated CD)' }; return { ok: true, warn: 'generic bootable ISO — boots via sanboot (emulated CD)' };
} }
// Remote-share ISOs aren't introspected (no random access over the // v0.7.4: NFS/SFTP ISOs now introspect over the share, so a probed
// network), so el_torito is unknown — assume bootable and let sanboot // remote ISO flows through the kernel/el_torito branches above like
// try rather than cry wolf. // a local one. introspect_rev 0 means the probe hasn't landed yet
// (it runs in the background right after a scan) or never can (SMB —
// smbclient can't seek): stay optimistic and let sanboot try.
const remote = iso.source && iso.source.kind && iso.source.kind !== 'local'; const remote = iso.source && iso.source.kind && iso.source.kind !== 'local';
if (remote) { if (remote && (iso.introspection.introspect_rev || 0) === 0) {
return { ok: true, warn: 'remote ISO — not introspected; sanboot is attempted at boot' }; return { ok: true, warn: 'remote ISO — awaiting introspection; sanboot is attempted at boot' };
} }
// Local ISO with no Windows/Linux boot files and no El Torito catalog: // Local ISO with no Windows/Linux boot files and no El Torito catalog:
// a data/appliance image (e.g. a VMware vCenter bundle), not a bootable // a data/appliance image (e.g. a VMware vCenter bundle), not a bootable
@@ -199,97 +201,47 @@
})[k] || (k || 'Unknown'); })[k] || (k || 'Unknown');
} }
// v0.7.0: the Boot rules card — ordered first-match-wins rules // v0.7.2: compact read-out of saved group rules — created from the
// (MAC prefix / architecture → target) plus the optional // unified "Pin MAC" form on the Hosts tab (a prefix or an architecture
// boot-decision webhook. Saved as one config because rule order // there saves a rule instead of a pin). First match wins, top to
// matters. With no rules and no webhook, behavior is identical to // bottom. The boot-decision webhook remains available via the API
// before the feature existed. // (/api/boot-rules `webhook_url`) but no longer has a UI knob.
function bootRulesCard(cfg, targetOptions) { function groupRulesCard(cfg, targetOptions) {
const archChoices = [ const rules = (cfg && cfg.rules) || [];
['', 'any arch'], ['bios', 'BIOS'], ['uefi-x64', 'UEFI x64'], if (!rules.length) return null;
['uefi-ia32', 'UEFI IA32'], ['uefi-arm64', 'UEFI ARM64'], const titleFor = id => {
]; const t = targetOptions.find(x => x.id === id);
const rules = (cfg.rules || []).map(r => Object.assign({}, r)); return t ? t.title : id;
const tbody = el('tbody', {});
const msg = el('div', {class:'msg'});
const webhookInput = el('input', {type:'text', spellcheck:'false',
placeholder:'http://automation.example/boot-decision (optional)',
value: cfg.webhook_url || ''});
const targetSelect = (val) => el('select', {},
[el('option', {value:''}, '— target —')]
.concat(targetOptions.map(t =>
el('option', Object.assign({value: t.id}, t.id === val ? {selected:''} : {}), t.title))));
const redraw = () => {
tbody.innerHTML = '';
if (!rules.length) {
tbody.appendChild(el('tr', {}, el('td', {colspan:'6', class:'empty', style:'padding:14px'},
'No rules. Add one to route whole groups of machines (an OUI, an architecture) to a target.')));
}
rules.forEach((r, i) => {
const macIn = el('input', {type:'text', spellcheck:'false', placeholder:'aa:bb:cc (prefix)',
value: r.mac_prefix || '', oninput: e => { r.mac_prefix = e.target.value; }});
const archSel = el('select', {onchange: e => { r.arch = e.target.value; }},
archChoices.map(([v, label]) =>
el('option', Object.assign({value: v}, v === (r.arch || '') ? {selected:''} : {}), label)));
const tgtSel = targetSelect(r.target || '');
tgtSel.onchange = e => { r.target = e.target.value; };
const noteIn = el('input', {type:'text', placeholder:'note',
value: r.note || '', oninput: e => { r.note = e.target.value; }});
const enabled = el('input', {type:'checkbox', onchange: e => { r.enabled = e.target.checked; }});
enabled.checked = r.enabled !== false;
tbody.appendChild(el('tr', {}, [
el('td', {}, macIn),
el('td', {}, archSel),
el('td', {}, tgtSel),
el('td', {}, noteIn),
el('td', {style:'text-align:center'}, enabled),
el('td', {style:'text-align:right'},
el('button', {class:'danger', onclick: () => { rules.splice(i, 1); redraw(); }}, '✕')),
]));
});
}; };
redraw(); const modeLabel = {firmware:'Firmware NIC', builtin:'iPXE drivers', shim:'Secure Boot (shim)'};
const rows = rules.map((r, i) => el('tr', r.enabled === false ? {style:'opacity:.5'} : {}, [
const addBtn = el('button', {class:'ghost', onclick: () => { el('td', {class:'mono'}, r.mac_prefix || el('span', {class:'tag'}, 'any MAC')),
rules.push({mac_prefix:'', arch:'', target:'', enabled:true, note:''}); el('td', {}, r.arch || el('span', {class:'tag'}, 'any arch')),
redraw(); el('td', {}, r.target ? titleFor(r.target) : el('span', {class:'tag'}, '—')),
}}, '+ Add rule'); el('td', {}, r.driver_mode
const saveBtn = el('button', {onclick: async () => { ? el('span', {class:'tag accent'}, modeLabel[r.driver_mode] || r.driver_mode)
const bad = rules.find(r => r.enabled !== false && !r.target); : el('span', {class:'tag'}, 'auto')),
if (bad) { msg.textContent = 'Every enabled rule needs a target.'; msg.className = 'msg err'; return; } el('td', {}, r.note || ''),
const r = await putJSON('/api/boot-rules', {rules, webhook_url: webhookInput.value.trim()}); el('td', {style:'text-align:right'},
if (r.ok) { msg.textContent = 'Saved.'; msg.className = 'msg ok'; } el('button', {class:'danger', onclick: async () => {
else { msg.textContent = 'Save failed: ' + await r.text(); msg.className = 'msg err'; } if (!confirm('Remove this group rule?')) return;
}}, 'Save rules'); const fresh = await getJSON('/api/boot-rules').catch(() => ({rules: [], webhook_url: ''}));
(fresh.rules = fresh.rules || []).splice(i, 1);
await putJSON('/api/boot-rules', fresh);
render('hosts');
}}, 'Remove')),
]));
return el('div', {class:'card'}, [ return el('div', {class:'card'}, [
el('header', {}, [ el('header', {}, [
el('h2', {}, 'Boot rules'), el('h2', {}, 'Group rules'),
el('span', {class:'sub'}, 'first match wins · checked top to bottom'), el('span', {class:'sub'}, 'first match wins · checked top to bottom'),
]), ]),
el('div', {class:'body'}, [ el('table', {}, [
el('table', {}, [ el('thead', {}, el('tr', {}, [
el('thead', {}, el('tr', {}, [ el('th',{},'MAC prefix'), el('th',{},'Arch'), el('th',{},'Target'),
el('th',{},'MAC prefix'), el('th',{},'Arch'), el('th',{},'Target'), el('th',{},'Boot binary'), el('th',{},'Note'), el('th',{},''),
el('th',{},'Note'), el('th',{},'On'), el('th',{},''), ])),
])), el('tbody', {}, rows),
tbody,
]),
el('div', {style:'margin-top:12px'}, [addBtn, saveBtn]),
el('label', {class:'field', style:'margin-top:16px;display:block'}, [
el('span', {class:'name'}, 'Boot-decision webhook (optional)'),
webhookInput,
el('span', {class:'hint'},
'When no pin or rule matches, OpenPXE GETs this URL with ?mac=…&arch=… ' +
'A 200 reply of {"target": "<entry-id>"} chains to that target; anything ' +
'else (404, timeout, error) falls through to the menu — a dead endpoint ' +
'can never block PXE.'),
]),
msg,
el('p', {class:'msg', style:'margin-top:10px'},
'Decision order per boot: exact MAC pin → first matching rule → webhook → interactive menu.'),
]), ]),
]); ]);
} }
@@ -317,7 +269,7 @@
el('label', {class:'field'}, [ el('label', {class:'field'}, [
el('span', {class:'name'}, 'Auto IP address (optional)'), ipInput]), el('span', {class:'name'}, 'Auto IP address (optional)'), ipInput]),
el('label', {class:'field'}, [ el('label', {class:'field'}, [
el('span', {class:'name'}, 'Unattended file'), sel]), el('span', {class:'name'}, 'Unattended file (in Storage → Advanced)'), sel]),
]); ]);
return { return {
wrap, wrap,
@@ -495,6 +447,12 @@
el('div', {class:'v'}, net.gateway || '?'), el('div', {class:'v'}, net.gateway || '?'),
el('div', {class:'k'}, 'Public base URL'), el('div', {class:'k'}, 'Public base URL'),
el('div', {class:'v'}, net.public_base_url), el('div', {class:'v'}, net.public_base_url),
// v0.7.2: physical link details (operstate · speed · duplex ·
// port MAC) so the operator can confirm WHICH port answers PXE
// in multi-NIC / trunked environments. Kept last — the joined
// value runs long, so it wraps cleanly at the bottom of the list.
el('div', {class:'k'}, 'Link'),
el('div', {class:'v'}, net.nic_link || '—'),
]), ]),
el('p', {class:'msg'}, el('p', {class:'msg'},
'Server IP, NIC, mask, and gateway are auto-detected at startup. ' + 'Server IP, NIC, mask, and gateway are auto-detected at startup. ' +
@@ -645,36 +603,85 @@
// ── Upload card ── // ── Upload card ──
const drop = el('div', {class:'drop', id:'drop'}, [ const drop = el('div', {class:'drop', id:'drop'}, [
el('div', {}, ['Drop an ', el('strong', {}, '.iso'), ' here, or click to choose.']), el('div', {}, ['Drop one or more ', el('strong', {}, '.iso'), ' files here, or click to choose.']),
el('div', {style:'font-size:12px;margin-top:6px'}, el('div', {style:'font-size:12px;margin-top:6px'},
'Linux + Windows installers auto-detected on upload. Streaming, no 502s on big files.'), 'Linux + Windows installers auto-detected on upload. Multiple files upload at once. Streaming, no 502s on big files.'),
]); ]);
const file = el('input', {type:'file', accept:'.iso,application/octet-stream', const file = el('input', {type:'file', accept:'.iso,application/octet-stream',
style:'display:none', id:'file'}); multiple:true, style:'display:none', id:'file'});
const prog = el('div', {class:'progress', id:'prog'}, el('div', {class:'bar', id:'bar'})); // v0.8.0: one progress row per file, appended here. Replaces the
const upMsg = el('div', {class:'msg', id:'upmsg'}); // single shared bar/msg/cancel that a second concurrent upload used
// v0.5.8: cancel button — shown only while an upload is in flight. // to clobber.
const cancelUpload = el('button', {class:'danger', type:'button', const uploadsList = el('div', {id:'uploads', style:'display:grid;gap:12px'});
style:'display:none;margin-top:12px', id:'cancel-upload'}, 'Cancel upload');
// One page-leave guard + one tab-hide cleanup for the whole card,
// registered only while ≥1 upload is in flight (added on 0→1, removed
// on 1→0) so listeners never pile up across re-renders.
let activeUploads = 0;
const activeIds = new Set();
const warnLeave = (e) => { if (activeUploads > 0) { e.preventDefault(); e.returnValue = ''; return ''; } };
const abortOnHide = () => {
// keepalive lets these DELETEs outlive the unload; the server also
// reclaims an orphaned .partial on the next upload, so best-effort
// is fine here.
for (const id of activeIds) {
try { fetch('/api/uploads/' + encodeURIComponent(id), {method:'DELETE', keepalive:true}); } catch (_) {}
}
};
const addGuards = () => {
window.addEventListener('beforeunload', warnLeave);
window.addEventListener('pagehide', abortOnHide);
};
const removeGuards = () => {
window.removeEventListener('beforeunload', warnLeave);
window.removeEventListener('pagehide', abortOnHide);
};
const failText = async (r) => {
const text = (await r.text()).slice(0, 240);
let hint = '';
if (r.status === 413) hint = ' - body too large. A proxy likely rejected this chunk.';
else if (r.status === 502) hint = ' - bad gateway. Proxy lost the upstream mid-stream.';
else if (r.status === 504) hint = ' - gateway timeout. Try the LAN IP directly.';
else if (r.status === 409) hint = ' - name conflict or offset mismatch. Remove the old ISO and retry.';
return 'HTTP ' + r.status + ' ' + text + hint;
};
// Launch an upload per dropped/selected .iso. The browser's ~6
// connections-per-origin cap naturally bounds how many stream at
// once, so there's no hand-rolled queue. Non-.iso files are ignored.
const startMany = (fileList) => {
[...fileList].filter(f => /\.iso$/i.test(f.name)).forEach(uploadOne);
};
drop.onclick = () => file.click(); drop.onclick = () => file.click();
drop.addEventListener('dragover', e => { e.preventDefault(); drop.classList.add('hover'); }); drop.addEventListener('dragover', e => { e.preventDefault(); drop.classList.add('hover'); });
drop.addEventListener('dragleave', () => drop.classList.remove('hover')); drop.addEventListener('dragleave', () => drop.classList.remove('hover'));
drop.addEventListener('drop', e => { drop.addEventListener('drop', e => {
e.preventDefault(); drop.classList.remove('hover'); e.preventDefault(); drop.classList.remove('hover');
if (e.dataTransfer.files[0]) upload(e.dataTransfer.files[0]); startMany(e.dataTransfer.files);
}); });
file.onchange = () => { if (file.files[0]) upload(file.files[0]); }; // Reset value so re-selecting the same filename still fires onchange.
file.onchange = () => { startMany(file.files); file.value = ''; };
// Chunked upload telemetry. The old browser path posted one huge // One independent chunked upload with its own progress row. The old
// multipart body, which left operators staring at 0% when a reverse // browser path posted one huge multipart body, which left operators
// proxy buffered or rejected the request before OpenPXE saw it. This // staring at 0% when a reverse proxy buffered or rejected the request
// path writes small raw chunks; each acknowledged chunk advances the // before OpenPXE saw it. This path writes small raw chunks; each
// bar and leaves a visible .partial file in the ISO directory. // acknowledged chunk advances the bar and leaves a visible .partial.
async function upload(f) { async function uploadOne(f) {
const started = Date.now(); const started = Date.now();
const bar = $('#bar'); const bar = el('div', {class:'bar'});
const setStatus = (text, cls) => { upMsg.textContent = text; upMsg.className = 'msg ' + (cls || ''); }; const prog = el('div', {class:'progress active'}, bar);
const rowMsg = el('div', {class:'msg'});
const cancelBtn = el('button', {class:'danger', type:'button', style:'margin-top:8px'}, 'Cancel');
const row = el('div', {}, [
el('div', {style:'font-weight:600;font-size:13px;margin-bottom:6px;word-break:break-all'}, f.name),
prog, rowMsg, cancelBtn,
]);
uploadsList.appendChild(row);
const setStatus = (text, cls) => { rowMsg.textContent = text; rowMsg.className = 'msg ' + (cls || ''); };
const update = (loaded, total, phase) => { const update = (loaded, total, phase) => {
const pct = total > 0 ? Math.min(100, (loaded / total) * 100) : 100; const pct = total > 0 ? Math.min(100, (loaded / total) * 100) : 100;
bar.style.width = pct.toFixed(1) + '%'; bar.style.width = pct.toFixed(1) + '%';
@@ -682,34 +689,24 @@
const rate = loaded > 0 ? loaded / elapsed : 0; const rate = loaded > 0 ? loaded / elapsed : 0;
const remain = rate > 0 ? (total - loaded) / rate : 0; const remain = rate > 0 ? (total - loaded) / rate : 0;
setStatus( setStatus(
phase + ' ' + f.name + ' - ' + phase + ' - ' +
fmtBytes(loaded) + ' of ' + fmtBytes(total) + fmtBytes(loaded) + ' of ' + fmtBytes(total) +
' (' + pct.toFixed(1) + '%, ' + fmtBytes(rate) + '/s' + ' (' + pct.toFixed(1) + '%, ' + fmtBytes(rate) + '/s' +
(remain > 0 ? ', ' + Math.ceil(remain) + 's left' : '') + ')'); (remain > 0 ? ', ' + Math.ceil(remain) + 's left' : '') + ')');
}; };
const failText = async (r) => {
const text = (await r.text()).slice(0, 240);
let hint = '';
if (r.status === 413) hint = ' - body too large. A proxy likely rejected this chunk.';
else if (r.status === 502) hint = ' - bad gateway. Proxy lost the upstream mid-stream.';
else if (r.status === 504) hint = ' - gateway timeout. Try the LAN IP directly.';
else if (r.status === 409) hint = ' - name conflict or offset mismatch. Remove the old ISO and retry.';
return 'HTTP ' + r.status + ' ' + text + hint;
};
let uploadId = null; let uploadId = null;
// v0.5.8: cancel + leave-page guard. The AbortController stops the // The AbortController stops this upload's in-flight chunk on Cancel.
// in-flight chunk; the beforeunload listener warns the operator // The card-level beforeunload guard (added while activeUploads > 0)
// that navigating away aborts the upload (the server-side partial // warns on navigation; the server reclaims an abandoned .partial on
// is then cleaned up by the DELETE in the catch below). // the next upload either way.
const ac = new AbortController(); const ac = new AbortController();
let canceled = false; let canceled = false;
const warnLeave = (e) => { e.preventDefault(); e.returnValue = ''; return ''; }; cancelBtn.onclick = () => { canceled = true; ac.abort(); };
window.addEventListener('beforeunload', warnLeave);
cancelUpload.style.display = ''; activeUploads += 1;
cancelUpload.onclick = () => { canceled = true; ac.abort(); }; if (activeUploads === 1) addGuards();
setStatus('Preparing upload for ' + f.name + ' (' + fmtBytes(f.size) + ')'); setStatus('Preparing ' + f.name + ' (' + fmtBytes(f.size) + ')');
prog.classList.add('active');
bar.style.width = '1%'; bar.style.width = '1%';
try { try {
@@ -720,6 +717,7 @@
if (!begin.ok) throw new Error(await failText(begin)); if (!begin.ok) throw new Error(await failText(begin));
const session = await begin.json(); const session = await begin.json();
uploadId = session.upload_id; uploadId = session.upload_id;
activeIds.add(uploadId);
const chunkSize = Math.max(1024 * 1024, Number(session.chunk_size || 8 * 1024 * 1024)); const chunkSize = Math.max(1024 * 1024, Number(session.chunk_size || 8 * 1024 * 1024));
let offset = Number(session.offset || 0); let offset = Number(session.offset || 0);
@@ -744,23 +742,29 @@
} while (!finished); } while (!finished);
setStatus('Uploaded and analyzed: ' + f.name + ' (' + fmtBytes(f.size) + ')', 'ok'); setStatus('Uploaded and analyzed: ' + f.name + ' (' + fmtBytes(f.size) + ')', 'ok');
render('storage');
} catch (err) { } catch (err) {
if (uploadId) { if (uploadId) {
try { await fetch('/api/uploads/' + encodeURIComponent(uploadId), {method: 'DELETE'}); } try { await fetch('/api/uploads/' + encodeURIComponent(uploadId), {method: 'DELETE'}); }
catch {} catch (_) {}
} }
if (canceled || (err && err.name === 'AbortError')) { if (canceled || (err && err.name === 'AbortError')) {
setStatus('Upload canceled — partial file discarded.', ''); setStatus('Canceled — partial file discarded.', '');
} else { } else {
setStatus('Upload failed: ' + (err && err.message ? err.message : String(err)), 'err'); setStatus('Upload failed: ' + (err && err.message ? err.message : String(err)), 'err');
} }
} finally { } finally {
window.removeEventListener('beforeunload', warnLeave); if (uploadId) activeIds.delete(uploadId);
cancelUpload.style.display = 'none'; cancelBtn.style.display = 'none';
cancelUpload.onclick = null;
prog.classList.remove('active'); prog.classList.remove('active');
if (!upMsg.className.includes('ok')) bar.style.width = '0'; activeUploads -= 1;
if (activeUploads === 0) {
removeGuards();
// Refresh the table to show the new image(s) — but only if the
// operator is still on Storage. isConnected goes false once
// render() swapped the view, so a mid-upload tab change won't
// yank them back here.
if (uploadsList.isConnected) render('storage');
}
} }
} }
@@ -884,6 +888,12 @@
render('storage'); render('storage');
}; };
// v0.7.2: searchable haystack for the list filter — filename,
// detected family, category, and source all match.
const searchText = [
i.filename, familyLabel(i.introspection.family), i.category || '',
isSmb ? 'smb' : isNfs ? 'nfs' : 'local', i.id,
].join(' ').toLowerCase();
const tr = el('tr', b.ok ? {} : {class: 'unbootable'}, [ const tr = el('tr', b.ok ? {} : {class: 'unbootable'}, [
el('td', {}, [ el('td', {}, [
el('div', {style:'display:flex;align-items:center;gap:8px'}, [ el('div', {style:'display:flex;align-items:center;gap:8px'}, [
@@ -928,8 +938,42 @@
}}, 'Remove'), }}, 'Remove'),
]), ]),
]); ]);
tr.dataset.search = searchText;
rowsAndEditors.push(tr, editorRow); rowsAndEditors.push(tr, editorRow);
}); });
// v0.7.2: client-side filter over the image table; v0.7.4: paged
// 5 at a time so a 50-image library doesn't become a scroll wall.
// Rows travel in (row, password-editor) pairs. One view function
// applies filter-then-page; editors close on any view change.
const ISO_PAGE_SIZE = 5;
let isoPage = 0;
const pagerInfo = el('span', {});
const prevBtn = el('button', {class:'ghost', onclick: () => { isoPage -= 1; applyIsoListView(); }}, ' Prev');
const nextBtn = el('button', {class:'ghost', onclick: () => { isoPage += 1; applyIsoListView(); }}, 'Next ');
function applyIsoListView() {
const q = isoSearch.value.trim().toLowerCase();
const visible = [];
for (let k = 0; k + 1 < rowsAndEditors.length; k += 2) {
const row = rowsAndEditors[k];
rowsAndEditors[k + 1].style.display = 'none';
row.style.display = 'none';
if (!q || (row.dataset.search || '').includes(q)) visible.push(row);
}
const pages = Math.max(1, Math.ceil(visible.length / ISO_PAGE_SIZE));
if (isoPage >= pages) isoPage = pages - 1;
if (isoPage < 0) isoPage = 0;
visible.slice(isoPage * ISO_PAGE_SIZE, (isoPage + 1) * ISO_PAGE_SIZE)
.forEach(r => { r.style.display = ''; });
pagerInfo.textContent = visible.length
? 'Showing ' + (isoPage * ISO_PAGE_SIZE + 1) + '' +
Math.min(visible.length, (isoPage + 1) * ISO_PAGE_SIZE) + ' of ' + visible.length
: 'No images match';
prevBtn.disabled = isoPage === 0;
nextBtn.disabled = isoPage >= pages - 1;
}
const isoSearch = el('input', {type:'search', placeholder:'Filter images by name, type, or source',
spellcheck:'false', oninput: () => { isoPage = 0; applyIsoListView(); }});
const isoTable = isos.length const isoTable = isos.length
? el('table', {}, [ ? el('table', {}, [
el('thead', {}, el('tr', {}, [ el('thead', {}, el('tr', {}, [
@@ -941,6 +985,13 @@
el('tbody', {}, rowsAndEditors), el('tbody', {}, rowsAndEditors),
]) ])
: el('div', {class:'empty'}, 'No images yet. Upload an ISO or add an SMB share.'); : el('div', {class:'empty'}, 'No images yet. Upload an ISO or add an SMB share.');
// v0.7.4: pager footer, shown once the library outgrows one page.
const isoPager = isos.length > ISO_PAGE_SIZE
? el('div', {class:'list-pager'}, [
pagerInfo, el('span', {class:'spacer'}), prevBtn, nextBtn,
])
: null;
if (isos.length) applyIsoListView();
// ── Remote shares section (v0.5.1) ── // ── Remote shares section (v0.5.1) ──
// SMB + NFS unified into one "Remote shares" card with a protocol // SMB + NFS unified into one "Remote shares" card with a protocol
@@ -1267,7 +1318,10 @@
unattFile.onchange = () => { if (unattFile.files[0]) uploadUnattended(unattFile.files[0]); }; unattFile.onchange = () => { if (unattFile.files[0]) uploadUnattended(unattFile.files[0]); };
const unattRows = unattendedFiles.length const unattRows = unattendedFiles.length
? unattendedFiles.map(f => el('div', {class:'nfs-row'}, [ ? unattendedFiles.map(f => el('div', {
class:'nfs-row',
'data-search': (f.filename + ' ' + unattendedKindLabel(f.kind) + ' ' + f.id).toLowerCase(),
}, [
el('span', {class:'dot ok'}), el('span', {class:'dot ok'}),
el('div', {}, [ el('div', {}, [
el('div', {class:'id'}, [ el('div', {class:'id'}, [
@@ -1286,6 +1340,36 @@
])) ]))
: [el('div', {class:'empty'}, 'No unattended files yet.')]; : [el('div', {class:'empty'}, 'No unattended files yet.')];
// v0.7.2: filter for big answer-file libraries; v0.7.5: paged 5 at
// a time, the same filter-then-page view the image table uses.
const UNATT_PAGE_SIZE = 5;
let unattPage = 0;
const unattPagerInfo = el('span', {});
const unattPrev = el('button', {class:'ghost', onclick: () => { unattPage -= 1; applyUnattListView(); }}, ' Prev');
const unattNext = el('button', {class:'ghost', onclick: () => { unattPage += 1; applyUnattListView(); }}, 'Next ');
const unattSearch = el('input', {type:'search', placeholder:'Filter files by name or kind',
spellcheck:'false', oninput: () => { unattPage = 0; applyUnattListView(); }});
function applyUnattListView() {
if (!unattendedFiles.length) return; // empty-state div carries no dataset
const q = unattSearch.value.trim().toLowerCase();
const visible = unattRows.filter(r => {
r.style.display = 'none';
return !q || (r.dataset.search || '').includes(q);
});
const pages = Math.max(1, Math.ceil(visible.length / UNATT_PAGE_SIZE));
if (unattPage >= pages) unattPage = pages - 1;
if (unattPage < 0) unattPage = 0;
visible.slice(unattPage * UNATT_PAGE_SIZE, (unattPage + 1) * UNATT_PAGE_SIZE)
.forEach(r => { r.style.display = ''; });
unattPagerInfo.textContent = visible.length
? 'Showing ' + (unattPage * UNATT_PAGE_SIZE + 1) + '' +
Math.min(visible.length, (unattPage + 1) * UNATT_PAGE_SIZE) + ' of ' + visible.length
: 'No files match';
unattPrev.disabled = unattPage === 0;
unattNext.disabled = unattPage >= pages - 1;
}
applyUnattListView();
const unattendedAdvanced = el('details', {class:'advanced-disclosure', style:'margin-top:18px'}, [ const unattendedAdvanced = el('details', {class:'advanced-disclosure', style:'margin-top:18px'}, [
el('summary', {class:'advanced-summary'}, 'Advanced'), el('summary', {class:'advanced-summary'}, 'Advanced'),
el('div', {class:'card', style:'margin-top:14px'}, [ el('div', {class:'card', style:'margin-top:14px'}, [
@@ -1295,7 +1379,15 @@
]), ]),
el('div', {class:'body'}, [ el('div', {class:'body'}, [
unattDrop, unattFile, unattMsg, unattDrop, unattFile, unattMsg,
unattendedFiles.length > 1
? el('label', {class:'field', style:'margin-top:14px;margin-bottom:0'}, unattSearch)
: null,
el('div', {style:'margin-top:16px;display:grid;gap:8px'}, unattRows), el('div', {style:'margin-top:16px;display:grid;gap:8px'}, unattRows),
unattendedFiles.length > UNATT_PAGE_SIZE
? el('div', {class:'list-pager', style:'padding:12px 0 0'}, [
unattPagerInfo, el('span', {class:'spacer'}), unattPrev, unattNext,
])
: null,
el('p', {class:'msg', style:'margin-top:14px'}, el('p', {class:'msg', style:'margin-top:14px'},
'These answer files drive unattended installs. Attach one to a ' + 'These answer files drive unattended installs. Attach one to a ' +
'host pin (Hosts tab) or a queued device (Queue → Profile); on ' + 'host pin (Hosts tab) or a queued device (Queue → Profile); on ' +
@@ -1309,7 +1401,7 @@
diskCard, diskCard,
el('div', {class:'card'}, [ el('div', {class:'card'}, [
el('header', {}, el('h2', {}, 'Upload ISO')), el('header', {}, el('h2', {}, 'Upload ISO')),
el('div', {class:'body'}, [drop, file, prog, upMsg, cancelUpload]), el('div', {class:'body'}, [drop, file, uploadsList]),
]), ]),
// v0.5.1: SMB + NFS unified into one "Remote shares" card with a // v0.5.1: SMB + NFS unified into one "Remote shares" card with a
// protocol dropdown. Backend endpoints are unchanged; this is a // protocol dropdown. Backend endpoints are unchanged; this is a
@@ -1341,7 +1433,11 @@
el('h2', {}, 'Available images'), el('h2', {}, 'Available images'),
el('span', {class:'sub'}, isos.length + ' image' + (isos.length === 1 ? '' : 's')), el('span', {class:'sub'}, isos.length + ' image' + (isos.length === 1 ? '' : 's')),
]), ]),
isos.length > 1
? el('div', {class:'list-search'}, el('label', {class:'field', style:'margin-bottom:0'}, isoSearch))
: null,
isoTable, isoTable,
isoPager,
]), ]),
]), unattendedAdvanced]); ]), unattendedAdvanced]);
}, },
@@ -1366,8 +1462,22 @@
{id: '_tools_menu', title: '↳ Tools menu (built-in)'}, {id: '_tools_menu', title: '↳ Tools menu (built-in)'},
]; ];
const macInput = el('input', {type:'text', placeholder:'aa:bb:cc:dd:ee:ff', spellcheck:'false'}); const macInput = el('input', {type:'text', placeholder:'aa:bb:cc:dd:ee:ff or aa:bb:cc', spellcheck:'false'});
const labelInput = el('input', {type:'text', placeholder:'optional, e.g. "rack-3 spine"'}); const labelInput = el('input', {type:'text', placeholder:'optional, e.g. "rack-3 spine"'});
// v0.7.2: the former separate "Boot rules" card folded into this
// form. A full MAC with no architecture saves a per-host pin
// exactly as before; a MAC *prefix* and/or an architecture saves a
// first-match-wins group rule instead. Same form, one mental model.
const archSel = el('select', {}, [
['', 'any (this exact MAC)'], ['bios', 'BIOS'], ['uefi-x64', 'UEFI x64'],
['uefi-ia32', 'UEFI IA32'], ['uefi-arm64', 'UEFI ARM64'],
].map(([v, t]) => el('option', {value: v}, t)));
// v0.7.1's boot-binary pin keeps its home here too (auto = let the
// escalation ladder learn; shim = known Secure Boot fleet).
const binSel = el('select', {}, [
['', 'auto (learn per machine)'], ['firmware', 'Firmware NIC'],
['builtin', 'iPXE drivers'], ['shim', 'Secure Boot (shim)'],
].map(([v, t]) => el('option', {value: v}, t)));
const targetSel = el('select', {}, const targetSel = el('select', {},
[el('option', {value:''}, '— choose a target —')] [el('option', {value:''}, '— choose a target —')]
.concat(reserved.map(t => el('option', {value: t.id}, t.title))) .concat(reserved.map(t => el('option', {value: t.id}, t.title)))
@@ -1380,21 +1490,40 @@
// hostname/IP templated into the served answer file. // hostname/IP templated into the served answer file.
const profileFields = buildProfileFields({}, unattendedFiles, 'form-row cols-3'); const profileFields = buildProfileFields({}, unattendedFiles, 'form-row cols-3');
const FULL_MAC = /^([0-9a-f]{2}[:-]){5}[0-9a-f]{2}$/i;
const upsertBtn = el('button', {onclick: async () => { const upsertBtn = el('button', {onclick: async () => {
if (!macInput.value || !targetSel.value) { const mac = macInput.value.trim();
msg.textContent = 'MAC and target are required.'; msg.className = 'msg err'; return; const isGroup = !!archSel.value || !!binSel.value || (mac !== '' && !FULL_MAC.test(mac));
if (!isGroup) {
// Exact-MAC pin — unchanged behavior.
if (!mac || !targetSel.value) {
msg.textContent = 'MAC and target are required.'; msg.className = 'msg err'; return;
}
const r = await postJSON('/api/hosts', Object.assign({
mac, target: targetSel.value, label: labelInput.value,
}, profileFields.read()));
if (r.ok) { msg.textContent = 'Saved.'; msg.className = 'msg ok'; render('hosts'); }
else { msg.textContent = 'Save failed: ' + await r.text(); msg.className = 'msg err'; }
return;
} }
const r = await postJSON('/api/hosts', Object.assign({ // Group rule (prefix and/or architecture). Per-host profile
mac: macInput.value, target: targetSel.value, label: labelInput.value, // fields don't apply to a group — they're per-machine values.
}, profileFields.read())); if (!targetSel.value && !binSel.value) {
if (r.ok) { msg.textContent = 'A group rule needs a target or a boot binary.'; msg.className = 'msg err'; return;
msg.textContent = 'Saved.'; msg.className = 'msg ok';
render('hosts');
} else {
const t = await r.text();
msg.textContent = 'Save failed: ' + t; msg.className = 'msg err';
} }
}}, 'Bind MAC to target'); const p = profileFields.read();
if (p.auto_hostname || p.auto_ip || p.unattended_file) {
msg.textContent = 'Auto-deploy fields are per-machine — clear them, or use a full MAC.'; msg.className = 'msg err'; return;
}
const cfg = await getJSON('/api/boot-rules').catch(() => ({rules: [], webhook_url: ''}));
(cfg.rules = cfg.rules || []).push({
mac_prefix: mac, arch: archSel.value, target: targetSel.value,
driver_mode: binSel.value, enabled: true, note: labelInput.value,
});
const r = await putJSON('/api/boot-rules', cfg);
if (r.ok) { msg.textContent = 'Group rule saved.'; msg.className = 'msg ok'; render('hosts'); }
else { msg.textContent = 'Save failed: ' + await r.text(); msg.className = 'msg err'; }
}}, 'Bind to target');
const rows = hosts.map(h => { const rows = hosts.map(h => {
// v0.5.0: Wake-on-LAN. Only shown for bound hosts (this whole // v0.5.0: Wake-on-LAN. Only shown for bound hosts (this whole
@@ -1451,23 +1580,32 @@
el('div', {class:'card'}, [ el('div', {class:'card'}, [
el('header', {}, el('h2', {}, 'Pin MAC to boot target')), el('header', {}, el('h2', {}, 'Pin MAC to boot target')),
el('div', {class:'body'}, [ el('div', {class:'body'}, [
// v0.7.3: MAC · Label · Architecture · Boot binary share one
// 4-up row so the controls line up across the page; the
// per-field guidance that used to sit under them moved into the
// note below to keep the inputs flush. Target spans full width
// on its own line beneath them.
el('div', {class:'form-row'}, [ el('div', {class:'form-row'}, [
el('label', {class:'field'}, [el('span', {class:'name'}, 'MAC address'), macInput]), el('label', {class:'field'}, [el('span', {class:'name'}, 'MAC address or prefix'), macInput]),
el('label', {class:'field'}, [el('span', {class:'name'}, 'Label (optional)'), labelInput]), el('label', {class:'field'}, [el('span', {class:'name'}, 'Label (optional)'), labelInput]),
el('label', {class:'field', style:'grid-column:1 / -1'}, [ el('label', {class:'field'}, [el('span', {class:'name'}, 'Architecture (optional)'), archSel]),
el('span', {class:'name'}, 'Target'), el('label', {class:'field'}, [el('span', {class:'name'}, 'Boot binary (optional)'), binSel]),
targetSel, ]),
el('span', {class:'hint'}, el('label', {class:'field', style:'margin-top:14px'}, [
'Built-in shortcuts skip the menu entirely. Per-ISO entries chain straight to the boot script.'), el('span', {class:'name'}, 'Target'),
]), targetSel,
]), ]),
el('div', {style:'margin-top:16px'}, profileFields.wrap), el('div', {style:'margin-top:16px'}, profileFields.wrap),
upsertBtn, msg, upsertBtn, msg,
el('p', {class:'msg', style:'margin-top:14px'}, el('p', {class:'msg', style:'margin-top:14px'},
'When a client with a bound MAC requests boot.ipxe, OpenPXE ' + 'A full MAC pins one machine; a MAC prefix (OUI) or an architecture ' +
'short-circuits past the interactive menu and chains directly. ' + 'saves a first-match group rule. Pin the boot binary to “shim” for ' +
'If an unattended file is selected, the matching kernel argument ' + 'Secure Boot racks — zero failed boot cycles. When a matching client ' +
'is injected and the hostname/IP are templated into the answer file.'), 'requests boot.ipxe, OpenPXE short-circuits past the interactive menu ' +
'and chains directly; decision order is exact MAC pin → first matching ' +
'group rule → menu. If an unattended file is selected on a pin, the ' +
'matching kernel argument is injected and the hostname/IP are templated ' +
'into the answer file.'),
]), ]),
]), ]),
el('div', {class:'card'}, [ el('div', {class:'card'}, [
@@ -1477,7 +1615,7 @@
]), ]),
table, table,
]), ]),
bootRulesCard(rulesCfg, reserved.concat(targets)), groupRulesCard(rulesCfg, reserved.concat(targets)),
el('div', {class:'card'}, [ el('div', {class:'card'}, [
el('header', {}, [ el('header', {}, [
el('h2', {}, 'Host log'), el('h2', {}, 'Host log'),
@@ -1640,7 +1778,7 @@
}, },
settings: async () => { settings: async () => {
const [status, me, sso, notify, docs] = await Promise.all([ const [status, me, sso, notify, docs, apiKey] = await Promise.all([
getJSON('/api/status'), getJSON('/api/status'),
getJSON('/api/me').catch(() => ({})), getJSON('/api/me').catch(() => ({})),
getJSON('/api/sso').catch(() => ({ getJSON('/api/sso').catch(() => ({
@@ -1650,6 +1788,7 @@
// fetches the notify config + API docs it needs too. // fetches the notify config + API docs it needs too.
getJSON('/api/notify').catch(() => ({ enabled:false, kind:'slack' })), getJSON('/api/notify').catch(() => ({ enabled:false, kind:'slack' })),
getJSON('/api/docs').catch(() => ({ groups: [] })), getJSON('/api/docs').catch(() => ({ groups: [] })),
getJSON('/api/api-key').catch(() => ({ key:'', header:'x-api-key' })),
]); ]);
// ── Account card (Forms admin credentials, v0.4.5). // ── Account card (Forms admin credentials, v0.4.5).
@@ -1963,7 +2102,7 @@
// into a collapsible disclosure beneath the core settings cards — // into a collapsible disclosure beneath the core settings cards —
// webhook/email notifications + the API reference. Keeps Settings // webhook/email notifications + the API reference. Keeps Settings
// clean by default while leaving the knobs one click away. // clean by default while leaving the knobs one click away.
const [notifyCard, apiCard] = views._advancedCards(notify, docs); const [notifyCard, apiCard] = views._advancedCards(notify, docs, apiKey);
const advanced = el('details', {class:'advanced-disclosure', style:'margin-top:18px'}, [ const advanced = el('details', {class:'advanced-disclosure', style:'margin-top:18px'}, [
el('summary', {class:'advanced-summary'}, 'Advanced'), el('summary', {class:'advanced-summary'}, 'Advanced'),
el('div', {class:'grid', style:'margin-top:14px'}, [notifyCard, apiCard]), el('div', {class:'grid', style:'margin-top:14px'}, [notifyCard, apiCard]),
@@ -1978,7 +2117,7 @@
// and the API reference. There is no longer an Advanced sidebar tab; // and the API reference. There is no longer an Advanced sidebar tab;
// the Settings view folds these into a collapsible disclosure and // the Settings view folds these into a collapsible disclosure and
// passes in the pre-fetched `notify` + `docs` payloads. // passes in the pre-fetched `notify` + `docs` payloads.
_advancedCards: (notify, docs) => { _advancedCards: (notify, docs, apiKey) => {
// ── Notification config ── // ── Notification config ──
const nMsg = el('div', {class:'msg', style:'margin-top:12px'}); const nMsg = el('div', {class:'msg', style:'margin-top:12px'});
@@ -2093,14 +2232,47 @@
]), ]),
]); ]);
// ── API reference (relocated from Settings) ── // ── API key + reference (relocated from Settings) ──
const groups = docs.groups || []; const groups = docs.groups || [];
// v0.8.0: operator API key. Paste into the `x-api-key` request
// header to drive /api/* from Postman / scripts without a browser
// session (full operator access). Read + rotate via /api/api-key.
const keyHeader = (apiKey && apiKey.header) || 'x-api-key';
const keyField = el('input', {type:'text', readonly:true,
value: (apiKey && apiKey.key) || '(unavailable)',
style:'width:100%;font-family:var(--mono)'});
const keyMsg = el('span', {class:'hint', style:'margin-left:10px'});
const copyKey = el('button', {class:'ghost', type:'button', onclick: async () => {
try { await navigator.clipboard.writeText(keyField.value); keyMsg.textContent = 'Copied to clipboard.'; }
catch { keyField.select(); keyMsg.textContent = 'Select the field and copy.'; }
}}, 'Copy');
const regenKey = el('button', {class:'danger', type:'button', style:'margin-left:8px',
onclick: async () => {
if (!confirm('Regenerate the API key? The current key stops working immediately and any client using it must be updated.')) return;
const r = await postJSON('/api/api-key/regenerate', {});
if (r.ok) { const j = await r.json(); keyField.value = j.key || ''; keyMsg.textContent = 'New key generated.'; }
else { keyMsg.textContent = 'Regenerate failed: ' + (await r.text()).slice(0, 120); }
}}, 'Regenerate');
const apiKeyBlock = el('div', {style:'padding:16px;border-bottom:1px solid var(--border)'}, [
el('label', {class:'field', style:'margin-bottom:10px'}, [
el('span', {class:'name'}, 'API key'),
keyField,
el('span', {class:'hint'}, [
'Send as the ', el('code', {}, keyHeader),
' request header to call the API from Postman or scripts — full operator access, so keep it secret.',
]),
]),
el('div', {}, [copyKey, regenKey, keyMsg]),
]);
const apiCard = el('div', {class:'card'}, [ const apiCard = el('div', {class:'card'}, [
el('header', {}, [ el('header', {}, [
el('h2', {}, 'API reference'), el('h2', {}, 'API'),
el('span', {class:'sub'}, el('span', {class:'sub'},
groups.reduce((n, g) => n + (g.endpoints || []).length, 0) + ' endpoints'), groups.reduce((n, g) => n + (g.endpoints || []).length, 0) + ' endpoints'),
]), ]),
apiKeyBlock,
el('div', {class:'api-ref'}, el('div', {class:'api-ref'},
groups.length groups.length
? groups.map(g => el('div', {class:'group'}, [ ? groups.map(g => el('div', {class:'group'}, [
@@ -2154,9 +2326,24 @@
el('div', {class:'about-hero'}, [ el('div', {class:'about-hero'}, [
el('h2', {}, 'OpenPXE'), el('h2', {}, 'OpenPXE'),
el('p', {class:'lead'}, el('p', {class:'lead'},
'Air-gapped network PXE boot, container-native, that anyone can run. ' + 'The network-boot platform for modern infrastructure. Drop in an ISO ' +
'No CDN calls, no telemetry, no surprise external dependencies — ship ' + 'and every machine on your network — BIOS, UEFI, Secure Boot — can ' +
'the image once, run it forever.'), 'boot it, image from it, and install unattended. One container, one ' +
'static binary, nothing installed on clients, nothing leaving your network.'),
el('div', {style:'display:grid;grid-template-columns:repeat(3,1fr);gap:14px;margin:18px 0'}, [
['Boot anything', 'Linux, Windows, hypervisors, rescue tools — uploaded ' +
'ISOs become menu entries automatically, served on demand from local ' +
'disk or your existing NFS, SMB, or SFTP libraries.'],
['Adapt to every machine', 'Per-machine boot intelligence: firmware quirks, ' +
'NIC driver fallback, and a Microsoft-signed Secure Boot chain are ' +
'negotiated automatically and remembered — no toggles, no client prep.'],
['Run it in production', 'SAML single sign-on, token-scoped answer files, ' +
'fleet routing rules, Wake-on-LAN, queued mass deployment, Prometheus ' +
'metrics. Built in Rust for boot infrastructure that cannot flinch.'],
].map(([h, body]) => el('div', {}, [
el('h3', {style:'margin:0 0 6px;font-size:13.5px'}, h),
el('p', {class:'msg', style:'font-size:12px;margin:0'}, body),
]))),
el('div', {class:'who'}, [ el('div', {class:'who'}, [
el('span', {}, 'Developer: '), el('strong', {}, 'Miles Ward'), el('br'), el('span', {}, 'Developer: '), el('strong', {}, 'Miles Ward'), el('br'),
el('span', {}, 'Version: '), el('strong', {}, status.version || '?'), el('br'), el('span', {}, 'Version: '), el('strong', {}, status.version || '?'), el('br'),
@@ -2167,15 +2354,16 @@
]), ]),
el('div', {style:'margin-top:18px'}, [updBtn, updMsg]), el('div', {style:'margin-top:18px'}, [updBtn, updMsg]),
el('p', {class:'msg', style:'margin-top:18px'}, el('p', {class:'msg', style:'margin-top:18px'},
'iPXE is an internal implementation detail. Everything the firmware ' + 'Private by design: no telemetry, no CDN calls, no runtime ' +
'executes is generated from the settings on these tabs — there is no ' + 'dependencies on the outside world. Air-gapped labs, customer sites ' +
'hand-written .ipxe path anywhere in this product.'), 'without internet, and locked-down OpenShift clusters run the same ' +
'image, the same way, indefinitely.'),
el('p', {class:'msg'}, el('p', {class:'msg'},
'Vision: a deployment-grade tool that works on first try in the most ' + 'Principled by default: OpenPXE never asks an operator to install ' +
'awkward environments — air-gapped labs, customer sites without ' + 'test-signed drivers, modify a clients trust store, or weaken ' +
'internet, OpenShift clusters with strict SCCs — without ever asking ' + 'Secure Boot. Everything the firmware executes is generated from the ' +
'an operator to install drivers signed with test certificates or to ' + 'settings on these tabs — there are no hand-written boot scripts to ' +
'flip "testsigning" on a target machine.'), 'maintain and no internals to learn.'),
]), ]),
]); ]);
+27 -51
View File
@@ -1,66 +1,42 @@
# docker-compose for local / homelab deployment. # OpenPXE — single-host / homelab deployment.
# #
# Two usage patterns: # One container: DHCP proxy + TFTP + iPXE chainload + HTTP (web UI, boot
# scripts, and ISO range streaming).
# #
# 1. Local MVP test — host network, proxy-DHCP off (don't fight your # OPENPXE_PUBLIC_IP=192.168.1.49 docker compose up -d
# existing DHCP server on the LAN), TFTP + HTTP exposed on the host:
# #
# docker compose up openpxe-dev # (or put OPENPXE_PUBLIC_IP in a .env file beside this one). That's this
# host's LAN IP, advertised to PXE clients so the iPXE URLs resolve —
# OpenPXE refuses to start rather than advertise an address clients can't
# reach, so compose errors out below if it's unset.
# #
# 2. Real PXE deployment — host network, proxy-DHCP on, runs on a box # Host networking is REQUIRED: DHCPDISCOVER is a broadcast, and Docker
# plugged into the PXE network: # bridges / CNI overlays don't forward it into containers. In host mode
# the container binds these ports directly on the host:
# #
# # First set OPENPXE_PUBLIC_IP to this host's LAN address in .env # udp/67 DHCP proxy udp/4011 PXE Boot Server
# docker compose up openpxe # udp/69 TFTP tcp/4200 web UI + HTTP boot assets
# #
# On Linux hosts, `network_mode: host` gives the container direct access to # Web UI: http://<this-host>:4200/
# the physical NIC — required for DHCP proxy because CNI overlays and Docker
# bridges do not forward DHCPDISCOVER broadcasts into containers.
#
# On macOS / Windows hosts, `network_mode: host` is limited — the daemon
# runs in a Linux VM (Colima/Docker Desktop) so the "host" network is the
# VM, not your Mac. Proxy-DHCP is not feasible on macOS; use `openpxe-dev`
# with published ports and set DHCP-MODE=disabled.
services: services:
# Real PXE deployment (Linux hosts).
openpxe: openpxe:
image: openpxe:0.1.0 image: gitea.milesward.dev/mward4/openpxe:latest
build: container_name: openpxe
context: .
dockerfile: deploy/docker/Dockerfile
restart: unless-stopped restart: unless-stopped
network_mode: host network_mode: host
# The binary carries cap_net_bind_service as a file capability, so it
# binds the low DHCP/TFTP ports as a non-root user — no privileged mode.
cap_add:
- NET_BIND_SERVICE
environment: environment:
# REQUIRED on multi-homed hosts. Set to this machine's LAN IP so the OPENPXE_PUBLIC_IP: ${OPENPXE_PUBLIC_IP:?set this to the host LAN IP, e.g. 192.168.1.49}
# advertised iPXE URLs actually resolve from the PXE clients. Without # Web UI + HTTP boot assets. 4200 keeps clear of anything on :80
# this, OpenPXE will refuse to start rather than advertise a # (an Unraid webGUI, a reverse proxy, …).
# loopback address that can't be reached. OPENPXE_HTTP_PORT: "4200"
OPENPXE_PUBLIC_IP: ${OPENPXE_PUBLIC_IP:?set this to the host LAN IP} # proxy = coexist with the LAN's existing DHCP server (recommended).
OPENPXE_DHCP_MODE: proxy OPENPXE_DHCP_MODE: proxy
OPENPXE_LOG: info OPENPXE_LOG: info
volumes: volumes:
- ./data/isos:/var/lib/openpxe/isos - ./data/isos:/var/lib/openpxe/isos # uploaded / seeded .iso files
- ./data/work:/var/lib/openpxe/work - ./data/work:/var/lib/openpxe/work # settings, share state, scratch
# Dev / MVP container: published ports, DHCP disabled, HTTP on 8080.
# Use this on laptops where you want to curl the API or UI without
# running an actual PXE chain.
openpxe-dev:
image: openpxe:0.1.0
build:
context: .
dockerfile: deploy/docker/Dockerfile
environment:
OPENPXE_PUBLIC_IP: ${OPENPXE_PUBLIC_IP:-127.0.0.1}
OPENPXE_DHCP_MODE: disabled
OPENPXE_HTTP_PORT: "8080"
OPENPXE_TFTP_PORT: "6969"
OPENPXE_DHCP_PORT: "6767"
OPENPXE_LOG: info,openpxe=debug
ports:
- "8080:8080/tcp"
- "6969:6969/udp"
volumes:
- ./data/isos:/var/lib/openpxe/isos
- ./data/work:/var/lib/openpxe/work