Stop the install dying on non-UTF-8 command output (#118)

* Stop the install dying on non-UTF-8 command output

efibootmgr prints what firmware gave it, and an HP ProBook's NVRAM held a legacy BBS entry whose device path was raw binary. `_read_efibootmgr` captured that with `text=True`, so Python decoded it strictly and a byte in a boot entry nothing here reads killed the whole "Installing Arch + Omarchy" phase. Two machines have hit it now, on 0x80 and 0xe0.

Every text capture in the orchestrator had the same exposure — partition labels reach lsblk and blkid the same way firmware strings reach efibootmgr — so they all go through one `capture()` that replaces what it cannot decode. The file reads here already did this; the command reads were the gap.

Replacing bytes has its own failure mode, and it is worse than the crash: a mangled UUID would be written into fstab and the kernel cmdline, and `_validate_pre_mounted_filesystems` reads it back from the same command, agrees with itself, and ships a machine that cannot find its root. So `require_text()` refuses a value the installed system depends on — UUIDs, device names, the btrfs device handed to mount — while the boot-entry labels stay tolerant on purpose: the decisions taken from those are advisory, and refusing a junk one would abort the install this change exists to let finish.

The regression test runs a stand-in efibootmgr emitting the reported byte sequence, so it exercises the decode rather than a mock. test_keyboard.py loads the module through the package now that keyboard.py imports a sibling.

Closes #116

🤖 Generated by Opus 5 in Claude Code. Reviewed by Codex XHigh.

Co-Authored-By: Codex XHigh <noreply@openai.com>

* Say accurately why the boot-entry parsing stays tolerant

The previous commit called the decisions taken from a firmware label "advisory". They are not: a label match selects entries for deletion and rewrites the boot order, and a missing one aborts validation. Staying tolerant is still right, for a reason the comment did not give — an entry's text is its label and its device path together, the junk lives in the path, and nothing mangled reaches efibootmgr from there anyway, since only four-digit ASCII boot numbers become arguments and a boot order is filtered through the entries that parsed. A label that loses a byte costs a stale entry left behind, not a broken system.

The test docstring credited one machine with a fixture that carries the bytes from two.

Co-Authored-By: Codex XHigh <noreply@openai.com>

---------

Co-authored-by: Codex XHigh <noreply@openai.com>
This commit is contained in:
OmarchybotandCodex XHigh authored and GitHub committed 2026-08-23 12:57:03 +02:00
1 parent 0c1333837f
commit df80b320cb
5 files changed
+210 -50

No files matched your search

@@ -0,0 +1,54 @@
r"""Reading text out of the commands the installer shells out to.
Firmware and filesystem metadata reach the orchestrator as command output, and
neither is obliged to be UTF-8: an HP ProBook's NVRAM held a legacy BBS entry
whose device path began `BBS(HD,\x80\x7f\xff\x04`, efibootmgr printed those
bytes verbatim, and subprocess's strict text decoding turned a boot entry
nothing here reads into a failed install.
So capture() replaces what it cannot decode, and the boot-entry parsing that
reads its output stays tolerant on purpose: an entry's text is its label and its
device path together, and the junk lives in the path, so refusing one would abort
the install this exists to let finish. Nothing mangled reaches efibootmgr from
there either way — only four-digit ASCII boot numbers become arguments, and a
boot order is filtered through the entries that parsed — so a label that loses a
byte costs a stale entry left behind rather than a broken system.
An identifier is the other case entirely: require_text() refuses a UUID or a
device name that came through mangled, because that one is written into fstab and
the kernel cmdline, or handed to mount.
"""
from __future__ import annotations
import subprocess
REPLACEMENT = "\ufffd"
def capture(cmd: list[str], *, check: bool = False) -> subprocess.CompletedProcess:
"""Run cmd and return it with stdout/stderr decoded, bad bytes replaced."""
return subprocess.run(
cmd,
check=check,
capture_output=True,
text=True,
errors="replace",
)
def require_text(value: str, what: str) -> str:
"""Refuse a value the installed system depends on if bytes were replaced in it.
Validation reads these back from the same command that wrote them, so a
mangled UUID agrees with itself and ships a machine that cannot find its
root at boot. Stopping here says why while someone is still watching.
"""
if REPLACEMENT in value:
raise RuntimeError(f"{what} is not valid text: {value!r}")
return value
def capture_identifier(cmd: list[str], what: str) -> str:
"""Capture a single value the installed system will depend on."""
return require_text(capture(cmd, check=True).stdout.strip(), what)
@@ -11,9 +11,10 @@ generated: nothing on an Omarchy system reads it.
from __future__ import annotations
import subprocess
from pathlib import Path
from .command import capture
def configure_keyboard(target: Path, language: str) -> bool:
"""Write the console keymap into a mounted target without booting it.
@@ -25,12 +26,7 @@ def configure_keyboard(target: Path, language: str) -> bool:
if not language.strip():
return True
result = subprocess.run(
["localectl", "--no-pager", "list-keymaps"],
check=False,
text=True,
capture_output=True,
)
result = capture(["localectl", "--no-pager", "list-keymaps"])
if result.returncode != 0:
detail = result.stderr.strip() or "localectl returned an error"
raise RuntimeError(f"Unable to list keyboard layouts: {detail}")
@@ -47,12 +43,7 @@ def configure_keyboard(target: Path, language: str) -> bool:
None,
)
result = subprocess.run(
["systemd-firstboot", f"--root={target}", f"--keymap={language}", "--force"],
check=False,
text=True,
capture_output=True,
)
result = capture(["systemd-firstboot", f"--root={target}", f"--keymap={language}", "--force"])
if result.returncode != 0:
detail = result.stderr.strip() or "systemd-firstboot returned an error"
raise RuntimeError(f"Unable to configure keyboard layout {language}: {detail}")
@@ -33,6 +33,7 @@ from dataclasses import replace
from pathlib import Path
from . import archinstall_adapter as arch
from .command import capture, capture_identifier, require_text
from .context import InstallContext
from .keyboard import configure_keyboard
from .ui import error, info
@@ -793,11 +794,7 @@ def verify_protected_mounts(ctx: InstallContext) -> None:
def _is_mountpoint(path: Path) -> bool:
res = subprocess.run(
["findmnt", "-rn", str(path)],
capture_output=True,
text=True,
)
res = capture(["findmnt", "-rn", str(path)])
return res.returncode == 0 and bool(res.stdout.strip())
@@ -811,11 +808,9 @@ def _btrfs_root_device(ctx: InstallContext) -> str:
def _blkid_uuid(device: str) -> str:
res = subprocess.run(
["blkid", "-s", "UUID", "-o", "value", device],
capture_output=True, text=True, check=True,
uuid = capture_identifier(
["blkid", "-s", "UUID", "-o", "value", device], f"the UUID of {device}"
)
uuid = res.stdout.strip()
if not uuid:
raise RuntimeError(f"blkid returned no UUID for {device}")
return uuid
@@ -828,11 +823,9 @@ def _esp_device(ctx: InstallContext) -> str:
boot = _boot_intent(ctx)
esp_mp = ctx.target / boot["esp_mount"].lstrip("/")
res = subprocess.run(
["findmnt", "-n", "-o", "SOURCE", str(esp_mp)],
capture_output=True, text=True, check=True,
dev = capture_identifier(
["findmnt", "-n", "-o", "SOURCE", str(esp_mp)], f"the ESP device at {esp_mp}"
)
dev = res.stdout.strip()
if not dev:
raise RuntimeError(f"could not resolve ESP device at {esp_mp}")
return dev
@@ -904,10 +897,7 @@ _BOOT_ORDER_RE = re.compile(r"^BootOrder:\s*(.*)$")
def _read_efibootmgr() -> dict:
res = subprocess.run(
["efibootmgr"],
capture_output=True, text=True, check=True,
)
res = capture(["efibootmgr"], check=True)
entries: dict[str, str] = {}
order: list[str] = []
for line in res.stdout.splitlines():
@@ -926,16 +916,14 @@ def _find_label_entries(entries: dict[str, str], needle: str) -> list[str]:
def _split_partition_device(part_dev: str) -> tuple[str, int]:
parent = subprocess.run(
["lsblk", "-ndo", "PKNAME", part_dev],
capture_output=True, text=True, check=True,
).stdout.strip()
parent = capture_identifier(
["lsblk", "-ndo", "PKNAME", part_dev], f"the parent disk of {part_dev}"
)
if not parent:
raise RuntimeError(f"could not find parent disk for {part_dev}")
part_num = subprocess.run(
["lsblk", "-ndo", "PARTN", part_dev],
capture_output=True, text=True, check=True,
).stdout.strip()
part_num = capture_identifier(
["lsblk", "-ndo", "PARTN", part_dev], f"the partition number of {part_dev}"
)
if not part_num:
raise RuntimeError(f"could not find partition number for {part_dev}")
return f"/dev/{parent}", int(part_num)
@@ -994,7 +982,7 @@ def _debug_run(ctx: InstallContext, cmd: list[str]) -> None:
if not _install_debug_enabled():
return
_debug_log(ctx, "+ " + " ".join(cmd))
proc = subprocess.run(cmd, check=False, text=True, capture_output=True)
proc = capture(cmd)
if proc.stdout:
with ctx.log_path.open("a", encoding="utf-8") as log:
for line in proc.stdout.splitlines():
@@ -1805,7 +1793,10 @@ def create_factory_snapshot(ctx: InstallContext) -> None:
info("› target root is not the @ subvolume; skipping factory snapshot")
return
device = (_findmnt_value(ctx.target, "SOURCE") or "").split("[")[0]
device = require_text(
(_findmnt_value(ctx.target, "SOURCE") or "").split("[")[0],
f"the btrfs device backing {ctx.target}",
)
if not device:
raise RuntimeError(f"could not determine the btrfs device backing {ctx.target}")
@@ -1862,10 +1853,7 @@ def _scrub_factory_snapshot(factory: Path) -> None:
def _findmnt_value(path: Path, column: str) -> str | None:
res = subprocess.run(
["findmnt", "-no", column, str(path)],
capture_output=True, text=True,
)
res = capture(["findmnt", "-no", column, str(path)])
value = res.stdout.strip()
return value if res.returncode == 0 and value else None