build: native arm64→x86_64-musl cross-compile (cargo-zigbuild), no QEMU

The Rust `build` stage previously ran the entire compiler under QEMU x86_64
emulation on the arm64 builder. That was ~15x slower (one crate took >20 min)
and the emulated gcc/linker intermittently SIGSEGV'd or hung mid-link
(observed again building v0.5.2).

Pin the stage to $BUILDPLATFORM (native arm64 on Apple Silicon, amd64 in CI)
and cross-compile to x86_64-unknown-linux-musl with cargo-zigbuild — zig cc
supplies the musl sysroot + linker. rustc runs natively; no emulation. Build
drops from ~30 min to a few minutes and is deterministic. Output is the same
fully static musl binary (verified: x86_64, not a dynamic executable, 0
OpenSSL strings).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
Miles Ward
2026-05-31 17:46:50 -04:00
co-authored by Claude Opus 4.8
parent 7adf5e2918
commit a906c47f53
+30 -26
View File
@@ -60,35 +60,41 @@ COPY deploy/ipxe/local/ deploy/ipxe/local/
RUN mkdir -p assets/ipxe && bash scripts/build-ipxe.sh /src/assets/ipxe
########## build openpxe ##########
FROM rust:${RUST_VERSION}-bookworm AS build
WORKDIR /src
# v0.4.5: build a fully static musl binary (matches Bootimus v0.1.70's
# move). The resulting `/openpxe` has no glibc dependency at all, which:
# - Lets the runtime stage be any Linux distro (we still ship Debian
# slim for the `samba` / `wimtools` / `nfs-common` shellouts, but a
# scratch/distroless variant becomes a one-line swap).
# - Cuts a class of "GLIBC_2.39 not found" surprises when running on
# older RHEL/Rocky hosts that don't match Debian 12's libc version.
# - Sidesteps cross-compilation snags (the binary is its own world).
# v0.5.2: cross-compile the Rust binary NATIVELY — no QEMU.
#
# x86_64-unknown-linux-musl is fully static by default (no extra
# RUSTFLAGS needed). musl-tools provides the linker.
# This stage is pinned to $BUILDPLATFORM (the builder's native arch — arm64
# on an Apple-Silicon Mac, amd64 in x86 CI), exactly like `ipxe-build`. The
# Rust compiler therefore runs at full native speed and emits an
# x86_64-unknown-linux-musl binary via `cargo-zigbuild`, which uses `zig cc`
# as the cross-linker (it bundles the musl sysroot for every target, so
# there's no fiddly cross-gcc toolchain to assemble).
#
# Why this replaced the old `FROM rust ... --platform=linux/amd64` build:
# that ran the *entire* compiler under QEMU x86_64 emulation on the arm64
# host. It was ~15x slower (a single crate took >20 min) and the emulated
# gcc/linker intermittently SIGSEGV'd or hung mid-link. Cross-compiling
# sidesteps emulation entirely — the build is minutes, not half an hour,
# and is deterministic.
#
# The output is still a fully static musl binary with no glibc dependency,
# so the runtime stage stays free to be any Linux distro.
FROM --platform=$BUILDPLATFORM rust:${RUST_VERSION}-bookworm AS build
WORKDIR /src
# zig (via the `ziglang` pip package — cargo-zigbuild auto-discovers it as
# `python3 -m ziglang`) supplies the x86_64 musl sysroot + linker.
# cargo-zigbuild is the thin cargo wrapper that wires zig in as the linker.
RUN apt-get update \
&& apt-get install -y --no-install-recommends musl-tools \
&& apt-get install -y --no-install-recommends python3 python3-pip \
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add x86_64-unknown-linux-musl
&& rustup target add x86_64-unknown-linux-musl \
&& pip3 install --no-cache-dir --break-system-packages ziglang \
&& cargo install --locked cargo-zigbuild
# Copy the whole workspace in one go. We used to do a two-pass "cache-prime
# with stubs, then real build" dance for dep-compile reuse; that turned out
# to silently serve stale stub binaries when cargo's fingerprint didn't
# notice the source swap. A single build is ~1.5 min longer on cold cache
# but guarantees the binary reflects the sources we copied.
# Do not copy rust-toolchain.toml into the image. The local workspace pins
# developer tooling, but inside Docker we intentionally use the Rust version
# selected by the base image. Copying rust-toolchain.toml with
# `channel = "stable"` makes rustup download a second full toolchain during
# `cargo build`, which is slow and can exhaust small Colima/CI disks.
# the build, which is slow and can exhaust small Colima/CI disks.
COPY Cargo.toml Cargo.lock ./
COPY crates/ crates/
# Baseline binaries (BIOS / i386 / wimboot), then overlay the
@@ -100,13 +106,11 @@ COPY --from=fetch /src/assets/ipxe /src/assets/ipxe
COPY --from=ipxe-build /src/assets/ipxe/snponly.efi /src/assets/ipxe/snponly.efi
COPY --from=ipxe-build /src/assets/ipxe/ipxe.efi /src/assets/ipxe/ipxe.efi
# Cache cargo registry + target across builds. The mtime touch is
# belt-and-suspenders: cargo occasionally misses mtime-only changes on
# networked FS; this forces a fingerprint check.
# Cache cargo registry + target across builds. `cargo zigbuild` runs the
# native rustc (fast) and links for x86_64-musl with zig — no emulation.
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/src/target,sharing=locked \
find crates -name '*.rs' -exec touch {} + && \
cargo build --release --target x86_64-unknown-linux-musl --bin openpxe && \
cargo zigbuild --release --target x86_64-unknown-linux-musl --bin openpxe && \
cp target/x86_64-unknown-linux-musl/release/openpxe /openpxe && \
ls -l /openpxe