Add safe Windows firmware extraction for Snapdragon laptops (#221)

* Add qcom-firmware-extract

Copy the vendor-signed Qualcomm firmware named by a Snapdragon laptop device tree from its Windows driver store into /usr/lib/firmware/updates.

* Select matching Qualcomm firmware variants

* Trim firmware extractor comments

* Keep firmware selection and initramfs updates safe on partial installs

* Run package regression suites in existing CI

* Match kernel firmware compression and search order

* Limit the CI change to the package test scripts

* Match firmware names exactly when updating the manifest

* Count compressed firmware only when the kernel can load it

* Scope the README's firmware claim to this package

* Find DSP device-tree images under their Windows names

The Surface Pro 11 device tree asks for adsp_dtb.mbn and cdsp_dtb.mbn,
but Windows ships them as adsp_dtbs.elf and cdsp_dtbs.elf, so
extraction left both DSP device-tree images missing. Look for the
Windows name when the device-tree name is not found. Exact names still
take precedence, and the existing companion matching still selects
between variants.

Validated with test.sh and on a Surface Pro 11 against its own Windows
driver store: 5 of 5 firmware files staged, up from 3, and the ADSP and
CDSP boot with the device-tree images taken from the .elf files.

* Let later firmware sources fill in what a stage lacks

Any stage manifest, even an empty one, made --stage exit as already
staged and made --install use only the stage, so Windows was never read
again. A --stage rerun now keeps the staged files and looks for the
rest, and --install scans Windows only for what the stage lacks. -d
still replaces both. A rerun without root, which cannot read Windows,
keeps the stage and succeeds as before. Each run reads the stage
manifest once.

Mount points honour QCOM_FW_ROOT so the tests can stub the partition
scan. Both root checks now take the user ID from QCOM_FW_TEST_EUID when
the tests set it, and the install-mode check no longer skips when
QCOM_FW_ROOT is set.

* Let board packages list firmware that must not come from Windows

Some device trees name firmware that should not be copied from Windows:
files a board package produces or holds back until they are tested, and
files the machine's boot firmware loads before Linux, which Windows
drivers do not carry.
On the Surface Laptop 8 these are its USB4 router image and SOCCP pair.

A board package can now list such names in
/usr/share/qcom-firmware-extract/provided.d/<compatible>.list. On a
machine with that compatible they are never searched for, installed or
reported by --list-missing, and the log names the list for each one. An
unreadable list or an entry containing a space is skipped with a
warning.

A copy an earlier --install made of a newly listed name stays in place:
the same bytes would not show that the file is still this tool's copy,
since a board's own tool may keep the same file there. The next
--install stops recording it and prints where it is, for the board
package or the user to remove.

Since --list-missing omits listed names, a board must not list firmware
that a hardware check relies on unless its package installs it before
the check runs.

* Find staged firmware by file name when its device-tree path changed

Since the stage became the first source, --install takes a staged file
only by its exact device-tree path. After a full-disk install the stage
is the only copy, so a device tree that files an image under another
path could no longer be served from it. After Windows, fall back to a
staged file with the same name that the stage manifest lists, but not
when Windows refused differing variants of it as ambiguous.

* Test that listed firmware never sends the extractor to the disks

With every name installed or listed, --list-missing prints nothing,
--stage and --install neither list nor mount partitions, and listed
names are not installed from Windows or the stage. A control run
without the list does scan. Listing the names again leaves the copies
it installed in place but stops recording them, and another missing
file still sends the extractor to Windows for that file alone.

* Read only internal disks from an installed system

--install mounts the Windows partitions whenever the stage lacks a
file, which after a full-disk install can be every later run, and it
mounted every NTFS partition, USB disks included. It now reads only
partitions on disks that lsblk reports as neither removable, hotplug
nor USB. --stage in the live session still reads all of them, internal
disks first, and -d takes a driver store from anywhere.

---------

Co-authored-by: Jimmy Van Veen <jimmy.van.veen@gmail.com>
Co-authored-by: Brandon Williams <turbine.bmw@gmail.com>
Co-authored-by: Jim Martin <jimmartin@gmail.com>
This commit is contained in:
authored and GitHub committed 2026-10-09 10:13:34 -04:00
1 parent 69c684305e
commit 9ea749835f
6 files changed
+1336

No files matched your search

+4
View File
@@ -71,4 +71,8 @@ jobs:
./tests/superwhisper-bin-install.sh
pacman -S --noconfirm --quiet rclone >/dev/null
./tests/publish-artifact.sh
shopt -s nullglob
for test in pkgbuilds/*/test.sh; do
bash "$test"
done
'
@@ -0,0 +1,3 @@
{
"source": "local"
}
+25
View File
@@ -0,0 +1,25 @@
# Maintainer: Jimmy Van Veen
# Copy vendor-signed Qualcomm firmware named by the running device tree.
pkgname=qcom-firmware-extract
pkgver=1
pkgrel=5
pkgdesc="Copy vendor-signed Qualcomm Snapdragon firmware from the machine's Windows installation"
arch=('any')
url="https://github.com/omacom/omarchy-pkgs"
license=('GPL-2.0-or-later')
depends=('bash' 'coreutils' 'diffutils' 'findutils' 'util-linux' 'xz' 'zstd')
optdepends=('limine-mkinitcpio-hook: rebuild the boot image after installing firmware')
options=('!debug')
source=(
'qcom-firmware-extract'
'README.package.md'
)
sha256sums=('SKIP' 'SKIP')
package() {
install -Dm755 qcom-firmware-extract "${pkgdir}/usr/bin/qcom-firmware-extract"
install -Dm644 README.package.md "${pkgdir}/usr/share/doc/${pkgname}/README.md"
install -dm755 "${pkgdir}/usr/share/${pkgname}/provided.d"
}
@@ -0,0 +1,91 @@
# qcom-firmware-extract
Copies the vendor-signed Qualcomm firmware a Snapdragon laptop needs, including
the GPU zap shader and audio/compute DSP images, from the owner's own Windows
installation on the same machine into `/usr/lib/firmware/updates/`.
The package itself contains no firmware. The files it copies never leave the
machine and are never downloaded. It is a temporary measure until laptop
vendors contribute complete firmware sets to linux-firmware.
## How it works
The device tree names every firmware file the kernel will ask for
(`firmware-name` properties). The tool keeps the names that are missing under
`/usr/lib/firmware{,/updates}`, finds each one by file name in
`Windows/System32/DriverStore/FileRepository` on an NTFS partition it can
mount read-only: with `--stage`, any such partition, and with `--install`,
only those on internal disks. A DSP device-tree image the device tree calls
`*_dtb.mbn` is also found under its Windows name, `*_dtbs.elf`, when the exact
name is missing. If Windows carries several variants and linux-firmware has a
companion image from the same device-tree node, the sibling image's hash
selects the compatible variant. Identical duplicates are accepted; differing
variants without a unique companion match are skipped. The result is
installed under `/usr/lib/firmware/updates/<name>`. The GPU zap shader is
added to the initramfs through `/etc/mkinitcpio.conf.d/qcom-firmware.conf`;
the DSP images are loaded from the root filesystem. What was installed, from
where, and its checksum is recorded in
`/var/lib/omarchy/qcom-firmware/manifest`.
Nothing is model-specific. A laptop whose firmware linux-firmware already
ships gets nothing copied; a machine without a device tree exits at once.
## Firmware that does not come from Windows
Some boards name firmware that must not be copied from Windows. The board
package may handle it itself, for example by processing a vendor driver file
or by holding back an image not yet tested on that board. Or the machine's
own boot firmware loads it before Linux starts, and Windows drivers do not
carry it as a file. The board package lists such names, one per line, in
`/usr/share/qcom-firmware-extract/provided.d/<compatible>.list`, where
`<compatible>` is one of the board's device-tree compatible strings. Blank
lines are ignored, and a `#` at the start of a line or after a space starts
a comment. An entry with a space inside, or a list that cannot be read, is
skipped with a warning. On that board, listed names are never searched for,
installed or reported missing, even when Windows or the stage holds a copy;
everything else is handled as usual.
A copy that an earlier `--install` put in `/usr/lib/firmware/updates` before
the name was listed stays there, and while it does, it loads before any other
copy. The tool never deletes it: matching bytes would not prove that the file
is still the one it installed, since a board's own tool may keep the same file
at that path. The next `--install` stops recording it in the manifest and
prints its path, so the board package, or the user, can remove it.
Because `--list-missing` leaves listed names out, a board must not list
firmware that a hardware check relies on unless its package installs that
firmware before the check runs. Omarchy's `install/hardware/qualcomm/firmware.sh`
looks for `adsp` in that output to decide whether to keep the DSP driver off,
so a listed ADSP image would bypass the check.
## When it runs
- **Installer, live session:** `qcom-firmware-extract --stage DIR` right
after the disk is chosen, before anything is written. A full-disk install
destroys the Windows partition the files come from, so this is the only
moment they can be read. The stage is copied into the target. Running
`--stage` again, for example with `-d`, adds the files earlier runs did
not find.
- **Installer, hardware setup:** `qcom-firmware-extract --install --no-rebuild`
from `install/hardware/qualcomm/firmware.sh`. Each file comes from the stage
if it holds one, otherwise from a Windows partition still on an internal
disk. A staged file that a later device tree names under another path is
found by file name, after Windows, unless Windows held variants it refused
as ambiguous. The installer builds the boot image once afterwards.
- **Installed system:** `sudo qcom-firmware-extract` uses the stage and scans
the internal disks again for what it lacks; when nothing is missing, it does
not touch the disks. `sudo qcom-firmware-extract -d /path/to/FileRepository`
instead takes any driver store you can mount, including one on a USB disk
(a Windows install of the *same model*: the files are tied to the vendor's
signing keys). It rebuilds the boot image; reboot afterwards.
BitLocker volumes cannot be read; turn BitLocker off in Windows first.
## Retirement
When linux-firmware ships a machine's vendor directory, the tool copies nothing
and still configures its GPU firmware for early display. When every supported
machine is covered, drop the package and prune the `/usr/lib/firmware/updates`
entries listed in the manifest.
Derived from Canonical's `qcom-firmware-extract` (GPL-2+).
+579
View File
@@ -0,0 +1,579 @@
#!/bin/bash
#
# Copy missing vendor-signed Qualcomm firmware from a Windows driver store.
# Required filenames come from the running device tree.
#
# Modes
# --stage DIR Live ISO, before the disk is touched: scan the Windows
# partitions and save the missing files under DIR (the
# installer copies DIR into the target). Missing firmware
# is reported without aborting staging. A rerun adds the
# files earlier runs did not find.
# --install Installed system or the installer's chroot: take each
# file from the stage (--stage-dir, default
# /var/lib/omarchy/firmware-stage), else from a Windows
# partition still on an internal disk, write them under
# /usr/lib/firmware/updates, record a manifest, add the
# zap shader to the initramfs, and rebuild the boot image
# (unless --no-rebuild). This is the default mode.
# --list-missing Print the device-tree firmware names not present under
# /usr/lib/firmware and not listed for this board (below),
# one per line. Needs no root.
# -d DIR Use DIR as the driver store (a Windows/System32/
# DriverStore/FileRepository directory, or any directory
# tree holding the files) instead of the Windows
# partitions. --install then ignores the stage.
#
# Files land in /usr/lib/firmware/updates to avoid package ownership conflicts.
#
# A board package lists named firmware that must not come from Windows on its
# board, one name per line, in /usr/share/qcom-firmware-extract/provided.d/
# <device-tree compatible>.list: files the package handles itself, and files
# the machine's own boot firmware loads before Linux starts. Names listed
# for one of this machine's compatibles are never searched for, installed or
# reported missing. When nothing else is missing, the disks are not touched.
# A copy that an earlier run installed before the name was listed is left in
# place; the next --install stops recording it and prints where it is. Since
# --list-missing leaves them out, a board must not list firmware that a
# hardware check relies on, such as the ADSP image Omarchy checks before it
# enables the DSP driver, unless its package installs that file before the
# check runs.
#
# Derived from Canonical's qcom-firmware-extract (Tobias Heider, GPL-2+):
# https://salsa.debian.org/debian/qcom-firmware-extract
# SPDX-License-Identifier: GPL-2.0-or-later
set -euo pipefail
DT_ROOT=${QCOM_FW_DT_ROOT:-/sys/firmware/devicetree/base}
FW_ROOT=${QCOM_FW_FIRMWARE_ROOT:-/usr/lib/firmware}
KERNEL_CONFIG=${QCOM_FW_KERNEL_CONFIG:-/proc/config.gz}
PROVIDED_DIR=${QCOM_FW_PROVIDED_DIR:-/usr/share/qcom-firmware-extract/provided.d}
# Destination prefix for the installed-system and mount paths (tests only).
ROOT=${QCOM_FW_ROOT:-}
# The effective user ID the root checks see (tests only).
CHECKED_EUID=${QCOM_FW_TEST_EUID:-$EUID}
STATE_DIR="$ROOT/var/lib/omarchy/qcom-firmware"
MANIFEST="$STATE_DIR/manifest"
INITCPIO_CONF="$ROOT/etc/mkinitcpio.conf.d/qcom-firmware.conf"
DEFAULT_STAGE_DIR="$ROOT/var/lib/omarchy/firmware-stage"
DRIVERSTORE_PATH="Windows/System32/DriverStore/FileRepository"
mode=install
stage_dir=""
driverstore=""
rebuild=yes
usage() {
sed -n '2,/^# SPDX/{/^# SPDX/d;s/^# \{0,1\}//p}' "$0"
}
while (($#)); do
case $1 in
--stage) mode=stage; stage_dir=${2:?--stage needs a directory}; shift ;;
--stage-dir) stage_dir=${2:?--stage-dir needs a directory}; shift ;;
--install) mode=install ;;
--list-missing) mode=list ;;
--no-rebuild) rebuild=no ;;
-d) driverstore=${2:?-d needs a directory}; shift ;;
-h | --help) usage; exit 0 ;;
*) echo "qcom-firmware-extract: unknown option: $1" >&2; usage >&2; exit 2 ;;
esac
shift
done
log() { printf 'qcom-firmware-extract: %s\n' "$*"; }
warn() { printf 'qcom-firmware-extract: %s\n' "$*" >&2; }
# ---------------------------------------------------------------------------
# 1. What does this machine ask for, and what is already there?
# ---------------------------------------------------------------------------
# Prints "<node-name> <firmware path>" for every firmware-name entry in the
# device tree. A property can hold several NUL-separated strings.
dt_firmware_names() {
local prop node name
[[ -d $DT_ROOT ]] || return 0
find "$DT_ROOT" -name firmware-name -type f -print0 2>/dev/null | sort -z |
while IFS= read -r -d '' prop; do
node=${prop%/firmware-name}
node=${node#"$DT_ROOT"/}
tr '\0' '\n' <"$prop" | while IFS= read -r name; do
[[ -n $name ]] && printf '%s %s\n' "$node" "$name"
done
done
}
# The kernel tries plain firmware in every directory, then zstd and xz if built
# with them. gzip is not an automatic firmware-loader fallback.
firmware_suffixes=('' .zst .xz)
if kernel_config=$(gzip -dc -- "$KERNEL_CONFIG" 2>/dev/null); then
firmware_suffixes=('')
if grep -qx 'CONFIG_FW_LOADER_COMPRESS_ZSTD=y' <<<"$kernel_config"; then
firmware_suffixes+=(.zst)
fi
if grep -qx 'CONFIG_FW_LOADER_COMPRESS_XZ=y' <<<"$kernel_config"; then
firmware_suffixes+=(.xz)
fi
fi
firmware_file() {
local name=$1 dir ext
for ext in "${firmware_suffixes[@]}"; do
for dir in "$FW_ROOT/updates" "$FW_ROOT"; do
if [[ -f "$dir/$name$ext" ]]; then
printf '%s\n' "$dir/$name$ext"
return 0
fi
done
done
return 1
}
# Names that board packages list for this machine as not from Windows.
declare -A provided=()
load_provided_names() {
local compatible list line
[[ -r $DT_ROOT/compatible && -d $PROVIDED_DIR ]] || return 0
while IFS= read -r compatible; do
[[ -n $compatible && $compatible != */* ]] || continue
list="$PROVIDED_DIR/$compatible.list"
[[ -f $list ]] || continue
if [[ ! -r $list ]]; then
warn "ignoring $list: it cannot be read"
continue
fi
while IFS= read -r line || [[ -n $line ]]; do
# A comment starts at a '#' that begins the line or follows a space.
line=" $line"
line=${line%%[[:space:]]#*}
line=${line#"${line%%[![:space:]]*}"}
line=${line%"${line##*[![:space:]]}"}
[[ -n $line ]] || continue
if [[ $line == *[[:space:]]* ]]; then
warn "ignoring '$line' in $list: a firmware name has no spaces"
continue
fi
provided[$line]=$compatible
done <"$list"
done < <(tr '\0' '\n' <"$DT_ROOT/compatible")
return 0
}
load_provided_names
declare -a missing_nodes=() missing_names=() provided_names=()
while read -r node name; do
if [[ -n ${provided[$name]-} ]]; then
provided_names+=("$name")
continue
fi
firmware_file "$name" >/dev/null && continue
missing_nodes+=("$node")
missing_names+=("$name")
done < <(dt_firmware_names)
if [[ $mode == list ]]; then
((${#missing_names[@]})) && printf '%s\n' "${missing_names[@]}"
exit 0
fi
if [[ ! -d $DT_ROOT ]]; then
log "no device tree on this machine, nothing to do"
exit 0
fi
for name in "${provided_names[@]}"; do
log "skipping $name: listed in $PROVIDED_DIR/${provided[$name]}.list"
done
if [[ $mode != install ]] && ((${#missing_names[@]} == 0)); then
log "every firmware file the device tree names is already installed${provided_names[*]:+ or listed for this board}"
exit 0
fi
# Root is needed to write under /usr/lib/firmware (and, below, to mount
# partitions); staging a -d driver store into a user directory needs neither.
if [[ $CHECKED_EUID != 0 && $mode == install ]]; then
warn "must run as root"
exit 1
fi
# ---------------------------------------------------------------------------
# 2. Where can they come from?
# ---------------------------------------------------------------------------
declare -a sources=() # driver stores to search together (-d or Windows)
declare -a mounts=() # temporary mounts to undo
stage_source="" # the installer's stage, used before the driver stores
declare -A manifest=() # the manifest this run updates, lines by firmware name
declare -A staged=() # names the stage manifest lists
cleanup() {
local m
for m in "${mounts[@]-}"; do
[[ -n $m ]] || continue
umount "$m" 2>/dev/null || true
rmdir "$m" 2>/dev/null || true
done
}
trap cleanup EXIT
# Mount Windows driver stores read-only, preferring internal disks. With
# "internal-only", removable and external disks are left alone.
scan_windows_partitions() {
local scope=${1:-all} dev removable hotplug fstype tran mp bitlocker="" skipped=""
if [[ $CHECKED_EUID != 0 ]]; then
warn "must run as root to read the Windows partitions"
exit 1
fi
local -a internal=() external=()
# read merges empty columns, so the always-set RM and HOTPLUG come first
# and an empty TRAN comes last.
while read -r dev removable hotplug fstype tran; do
[[ $fstype == ntfs || $fstype == BitLocker ]] || continue
if [[ $removable == 0 && $hotplug == 0 && $tran != usb ]]; then
if [[ $fstype == ntfs ]]; then internal+=("$dev"); else bitlocker+=" $dev"; fi
elif [[ $scope == internal-only ]]; then
skipped+=" $dev"
elif [[ $fstype == ntfs ]]; then
external+=("$dev")
else
bitlocker+=" $dev"
fi
done < <(lsblk -rpno PATH,RM,HOTPLUG,FSTYPE,TRAN 2>/dev/null)
[[ -z $skipped ]] || log "not reading removable or external disk partition(s):$skipped; use -d for a driver store on one"
for dev in "${internal[@]}" "${external[@]}"; do
mp=$(mktemp -d "$ROOT/run/qcom-firmware-extract.XXXXXX")
if mount -t ntfs3 -o ro,nosuid,nodev,noexec "$dev" "$mp" 2>/dev/null; then
mounts+=("$mp")
if [[ -d "$mp/$DRIVERSTORE_PATH" ]]; then
log "Windows driver store found on $dev"
sources+=("$mp/$DRIVERSTORE_PATH")
fi
else
warn "could not mount $dev read-only (ntfs3); skipping it"
rmdir "$mp" 2>/dev/null || true
fi
done
[[ -n $bitlocker ]] && warn "BitLocker volume(s) skipped:$bitlocker; turn BitLocker off in Windows to let the firmware be read"
return 0
}
firmware_sha256() {
local file=$1
case "$file" in
*.zst) zstd -qdc -- "$file" ;;
*.xz) xz -dc -- "$file" ;;
*) cat -- "$file" ;;
esac | sha256sum | cut -d' ' -f1
}
# Match ambiguous DTBs to an installed sibling firmware blob from the same node.
matching_companion_candidate() {
local node=$1
shift
local -a candidates=("$@")
local dt_node dt_name installed installed_hash candidate sibling matched compatible
local -a matches=()
for candidate in "${candidates[@]}"; do
matched=0
compatible=1
while read -r dt_node dt_name; do
[[ $dt_node == "$node" ]] || continue
installed=$(firmware_file "$dt_name") || continue
installed_hash=$(firmware_sha256 "$installed") || return 1
sibling=$(find "${candidate%/*}" -maxdepth 1 -type f \
-iname "$(basename "$dt_name")" -print -quit 2>/dev/null)
[[ -n $sibling ]] || continue
if [[ $(sha256sum "$sibling" | cut -d' ' -f1) == "$installed_hash" ]]; then
matched=1
else
compatible=0
break
fi
done < <(dt_firmware_names)
if ((matched && compatible)); then
matches+=("$candidate")
fi
done
unique_candidate "${matches[@]}"
}
# Duplicate copies are safe only when their contents agree.
unique_candidate() {
(($#)) || return 1
local first=$1 candidate
shift
for candidate in "$@"; do
cmp -s "$first" "$candidate" || return 1
done
printf '%s\n' "$first"
}
# Windows can ship DSP device-tree images under another name: a device tree
# may ask for adsp_dtb.mbn where Windows has adsp_dtbs.elf (e.g. Surface Pro 11).
windows_firmware_names() {
printf '%s\n' "$1"
if [[ $1 == *_dtb.mbn ]]; then
printf '%s\n' "${1%_dtb.mbn}_dtbs.elf"
fi
}
# Require a companion match when several firmware variants exist. Returns 1
# when the sources have no candidate and 2 when it refuses ambiguous ones.
find_in_sources() {
local base=$1 node=$2 candidate name
local -a candidates=()
while IFS= read -r name; do
while IFS= read -r -d '' candidate; do
candidates+=("$candidate")
done < <(find "${sources[@]}" -type f -iname "$name" -print0 2>/dev/null)
((${#candidates[@]})) && break
done < <(windows_firmware_names "$base")
((${#candidates[@]})) || return 1
if unique_candidate "${candidates[@]}"; then
return 0
fi
if matching_companion_candidate "$node" "${candidates[@]}"; then
return 0
fi
warn "refusing ambiguous $base: no unique variant matches its installed companion firmware"
return 2
}
# A manifest has one line per firmware name: the name, the SHA-256 of the file
# and where it came from. Loads FILE as the manifest this run updates.
read_manifest() {
local line name
manifest=()
while IFS= read -r line || [[ -n $line ]]; do
read -r name _ <<<"$line"
if [[ -n $name ]]; then
manifest[$name]=$line
fi
done <"$1"
}
# Sets NAME's manifest line to NAME followed by FIELDS; without FIELDS, removes
# it.
set_manifest_line() {
local name=$1
shift
if (($#)); then
manifest[$name]="$name $*"
else
unset 'manifest[$name]'
fi
}
write_manifest() {
if ((${#manifest[@]})); then
printf '%s\n' "${manifest[@]}" | sort >"$1.new"
else
: >"$1.new"
fi
mv "$1.new" "$1"
}
# A file in DIR counts as staged when the stage manifest lists it. In --stage
# mode, that manifest is also the one the run updates.
use_stage() {
local name
stage_source=$1
read_manifest "$1/manifest"
for name in "${!manifest[@]}"; do
staged[$name]=1
done
}
is_staged() {
[[ -n ${staged[$1]-} && -f $stage_source/$1 ]]
}
# True if the stage lacks any of the missing files.
stage_incomplete() {
local name
for name in "${missing_names[@]}"; do
is_staged "$name" || return 0
done
return 1
}
# The stage comes first; the driver stores supply what it lacks. A staged file
# listed under another device-tree path with the same file name (a later device
# tree may move it) is the last resort, since after a full-disk install the
# stage is the only copy.
locate_firmware() {
local name=$1 node=$2 src status=0
if is_staged "$name"; then
printf '%s\n' "$stage_source/$name"
return 0
fi
if ((${#sources[@]})); then
src=$(find_in_sources "$(basename "$name")" "$node") || status=$?
if ((status == 0)); then
printf '%s\n' "$src"
return 0
fi
# The stage does not override a refusal of ambiguous variants.
((status != 2)) || return 1
fi
[[ -n $stage_source ]] || return 1
local -a sources=("$stage_source")
src=$(find_in_sources "$(basename "$name")" "$node") || return 1
is_staged "${src#"$stage_source"/}" || return 1
printf '%s\n' "$src"
}
# ---------------------------------------------------------------------------
# 3. Stage: live session, before the disk is written
# ---------------------------------------------------------------------------
if [[ $mode == stage ]]; then
# Files staged by an earlier run are kept; a rerun, for example with -d,
# looks only for the rest, so an empty or partial stage blocks nothing.
if [[ -f "$stage_dir/manifest" ]]; then
use_stage "$stage_dir"
fi
if ! stage_incomplete; then
log "already staged in $stage_dir"
exit 0
fi
if [[ -n $driverstore ]]; then
sources=("$driverstore")
elif [[ -n $stage_source && $CHECKED_EUID != 0 ]]; then
# Without root the partitions cannot be read; keep the stage as it is.
log "already staged in $stage_dir; run as root to read the Windows partitions for the rest"
exit 0
else
scan_windows_partitions
fi
mkdir -p "$stage_dir"
copied=0
kept=0
for i in "${!missing_names[@]}"; do
node=${missing_nodes[$i]}
name=${missing_names[$i]}
if is_staged "$name"; then
kept=$((kept + 1))
continue
fi
src=""
if ((${#sources[@]})); then
src=$(find_in_sources "$(basename "$name")" "$node") || true
fi
if [[ -z $src ]]; then
warn "not found in the driver store: $name"
continue
fi
install -Dm644 "$src" "$stage_dir/$name"
set_manifest_line "$name" "$(sha256sum "$src" | cut -d' ' -f1)" "$src"
copied=$((copied + 1))
done
write_manifest "$stage_dir/manifest"
log "staged $copied of ${#missing_names[@]} firmware files in $stage_dir ($kept already staged)"
exit 0
fi
# ---------------------------------------------------------------------------
# 4. Install: from the stage, else from a Windows partition still on disk
# ---------------------------------------------------------------------------
[[ -n $stage_dir ]] || stage_dir=$DEFAULT_STAGE_DIR
if [[ -n $driverstore ]]; then
sources=("$driverstore")
else
# Windows supplies what the stage lacks, so a stage never hides it.
if [[ -f "$stage_dir/manifest" ]]; then
log "using the firmware staged by the installer in $stage_dir"
use_stage "$stage_dir"
fi
# After installation, only internal disks are read; -d reads any other.
if stage_incomplete; then
scan_windows_partitions internal-only
fi
fi
mkdir -p "$STATE_DIR"
touch "$MANIFEST"
read_manifest "$MANIFEST"
# A board may list a name after an earlier run installed it. That copy still
# loads first, but this tool never deletes it: matching bytes would not prove
# that the file is still the one it installed, since a board's own tool may
# keep the same file there. Stop recording it and say where it is.
for name in "${provided_names[@]}"; do
[[ -n ${manifest[$name]-} ]] || continue
set_manifest_line "$name"
write_manifest "$MANIFEST"
file="$FW_ROOT/updates/$name"
if [[ -e $file || -L $file ]]; then
log "note: an earlier run installed $file, but $name is now listed in $PROVIDED_DIR/${provided[$name]}.list; the file is left in place and no longer recorded"
log "note: while it is there it loads before any other copy. The board package may remove it, or you can by hand. This tool does not, because matching bytes would not prove that the file is still the one it installed: a board's own tool may keep the same file there."
fi
done
installed=0
for i in "${!missing_names[@]}"; do
node=${missing_nodes[$i]}
name=${missing_names[$i]}
src=$(locate_firmware "$name" "$node") || true
if [[ -z $src ]]; then
warn "not found: $name"
continue
fi
install -Dm644 "$src" "$FW_ROOT/updates/$name"
# Record each file as soon as it is in place.
set_manifest_line "$name" "$(sha256sum "$src" | cut -d' ' -f1)" "$src" \
"$(date -u +%Y-%m-%dT%H:%M:%SZ)"
write_manifest "$MANIFEST"
log "installed $name"
installed=$((installed + 1))
done
# Include existing GPU zap shaders in the initramfs for the early DRM driver.
declare -a initramfs_files=()
while read -r node name; do
[[ ${node##*/} == zap-shader || ${node##*/} == zap-shader@* ]] || continue
if file=$(firmware_file "$name"); then
initramfs_files+=("${file#"$ROOT"}")
fi
done < <(dt_firmware_names)
initramfs_changed=0
if ((${#initramfs_files[@]})); then
initramfs_config=$(
echo "# Generated by qcom-firmware-extract for early DRM firmware loading."
printf 'FILES+=(%s)\n' "$(printf '%q ' "${initramfs_files[@]}" | sed 's/ $//')"
)
if [[ ! -f $INITCPIO_CONF || $(<"$INITCPIO_CONF") != "$initramfs_config" ]]; then
mkdir -p "$(dirname "$INITCPIO_CONF")"
printf '%s\n' "$initramfs_config" >"$INITCPIO_CONF"
initramfs_changed=1
fi
elif [[ -f $INITCPIO_CONF ]]; then
rm -f "$INITCPIO_CONF"
initramfs_changed=1
fi
still_missing=$(( ${#missing_names[@]} - installed ))
if ((installed == 0)); then
if ((still_missing > 0 && ${#sources[@]} == 0)); then
warn "no Windows installation found to copy from; run again with -d <driver store> when one is available"
fi
log "nothing installed; $still_missing firmware file(s) still missing"
((initramfs_changed)) || exit 0
else
log "installed $installed firmware file(s), $still_missing still missing"
fi
if [[ $rebuild == yes ]]; then
if command -v limine-update >/dev/null 2>&1; then
limine-update
else
mkinitcpio -P
fi
log "reboot to load the added firmware"
fi
+634
View File
@@ -0,0 +1,634 @@
#!/bin/bash
set -euo pipefail
package_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
extractor="$package_dir/qcom-firmware-extract"
scratch=$(mktemp -d)
trap 'rm -rf "$scratch"' EXIT
# macOS' install(1) gives -D a different meaning. Prefer GNU coreutils when
# this focused test runs on a contributor's Mac; Arch uses GNU install already.
test_bin="$scratch/bin"
mkdir -p "$test_bin"
if command -v ginstall >/dev/null 2>&1; then
ln -s "$(command -v ginstall)" "$test_bin/install"
fi
dt_root="$scratch/device-tree"
firmware_root="$scratch/firmware"
driver_store="$scratch/DriverStore"
stage="$scratch/stage"
node="$dt_root/remoteproc@0"
firmware_path="qcom/x1e80100/LENOVO/83ED"
# Pin the kernel's firmware decompressors instead of reading /proc/config.gz.
export QCOM_FW_KERNEL_CONFIG="$scratch/config.gz"
set_kernel_config() { printf '%s\n' "$@" | gzip >"$QCOM_FW_KERNEL_CONFIG"; }
set_kernel_config CONFIG_FW_LOADER_COMPRESS_ZSTD=y CONFIG_FW_LOADER_COMPRESS_XZ=y
mkdir -p "$node" "$firmware_root/$firmware_path" \
"$driver_store/wrong" "$driver_store/matching"
printf '%s\0%s\0' \
"$firmware_path/qccdsp8380.mbn" \
"$firmware_path/cdsp_dtbs.elf" >"$node/firmware-name"
# Make the incompatible Windows firmware newer than the matching variant.
printf 'installed-cdsp' >"$firmware_root/$firmware_path/qccdsp8380.mbn"
printf 'other-cdsp' >"$driver_store/wrong/qccdsp8380.mbn"
printf 'wrong-dtb' >"$driver_store/wrong/cdsp_dtbs.elf"
printf 'installed-cdsp' >"$driver_store/matching/qccdsp8380.mbn"
printf 'matching-dtb' >"$driver_store/matching/cdsp_dtbs.elf"
touch -t 203001010000 "$driver_store/wrong/cdsp_dtbs.elf"
touch -t 202001010000 "$driver_store/matching/cdsp_dtbs.elf"
QCOM_FW_DT_ROOT="$dt_root" \
QCOM_FW_FIRMWARE_ROOT="$firmware_root" \
PATH="$test_bin:$PATH" \
bash "$extractor" --stage "$stage" -d "$driver_store"
[[ $(<"$stage/$firmware_path/cdsp_dtbs.elf") == matching-dtb ]] || {
echo "not ok - extractor did not select the DTB matching the installed DSP image" >&2
exit 1
}
echo "ok - extractor selects an ambiguous DTB by its companion firmware hash"
# Runs under a test root prefix, as if root.
run_extractor() {
QCOM_FW_DT_ROOT="$dt_root" \
QCOM_FW_FIRMWARE_ROOT="$firmware_root" \
QCOM_FW_ROOT="$scratch/root" \
QCOM_FW_TEST_EUID=0 \
PATH="$test_bin:$PATH" \
bash "$extractor" "$@"
}
# A different remote processor must not supply the matching companion.
mkdir -p "$dt_root/remoteproc@1" "$driver_store/unrelated"
printf '%s\0' "$firmware_path/qcadsp8380.mbn" >"$dt_root/remoteproc@1/firmware-name"
printf 'installed-adsp' >"$firmware_root/$firmware_path/qcadsp8380.mbn"
printf 'installed-adsp' >"$driver_store/unrelated/qcadsp8380.mbn"
printf 'unrelated-dtb' >"$driver_store/unrelated/cdsp_dtbs.elf"
printf 'missing-cdsp' >"$firmware_root/$firmware_path/qccdsp8380.mbn"
run_extractor --stage "$scratch/unmatched" -d "$driver_store"
[[ ! -e $scratch/unmatched/$firmware_path/cdsp_dtbs.elf ]]
echo "ok - companion matching stays within the exact device-tree node"
# Firmware not found in Windows must not stop other files from being staged.
printf '%s\0%s\0%s\0' "$firmware_path/not-in-windows.mbn" \
"$firmware_path/cdsp_dtbs.elf" "$firmware_path/qccdsp8380.mbn" >"$node/firmware-name"
printf 'installed-cdsp' >"$firmware_root/$firmware_path/qccdsp8380.mbn"
run_extractor --stage "$scratch/partial" -d "$driver_store"
[[ $(<"$scratch/partial/$firmware_path/cdsp_dtbs.elf") == matching-dtb ]]
run_extractor --install --no-rebuild --stage-dir "$scratch/partial"
[[ $(<"$firmware_root/updates/$firmware_path/cdsp_dtbs.elf") == matching-dtb ]]
echo "ok - missing firmware does not abort staging or installation"
# Windows names the Surface Pro 11's DSP device-tree images *_dtbs.elf.
surface_path="qcom/x1e80100/microsoft/Denali"
mkdir -p "$driver_store/surfacepro_ext_adsp8380"
printf '%s\0' "$surface_path/adsp_dtb.mbn" >"$node/firmware-name"
printf 'surface-adsp-dtb' >"$driver_store/surfacepro_ext_adsp8380/adsp_dtbs.elf"
run_extractor --stage "$scratch/windows-name" -d "$driver_store"
[[ $(<"$scratch/windows-name/$surface_path/adsp_dtb.mbn") == surface-adsp-dtb ]] || {
echo "not ok - extractor did not find a DSP device-tree image under its Windows name" >&2
exit 1
}
run_extractor --install --no-rebuild --stage-dir "$scratch/windows-name"
[[ $(<"$firmware_root/updates/$surface_path/adsp_dtb.mbn") == surface-adsp-dtb ]]
rm "$firmware_root/updates/$surface_path/adsp_dtb.mbn"
printf 'exact-name' >"$driver_store/surfacepro_ext_adsp8380/adsp_dtb.mbn"
run_extractor --stage "$scratch/exact-name" -d "$driver_store"
[[ $(<"$scratch/exact-name/$surface_path/adsp_dtb.mbn") == exact-name ]]
rm "$driver_store/surfacepro_ext_adsp8380/adsp_dtb.mbn"
echo "ok - DSP device-tree images are found under their Windows names, exact names first"
# With no companion, only byte-identical duplicates are safe to select.
printf '%s\0' "$firmware_path/duplicate.mbn" >"$node/firmware-name"
printf 'one' >"$driver_store/wrong/duplicate.mbn"
printf 'two' >"$driver_store/matching/duplicate.mbn"
run_extractor --stage "$scratch/ambiguous" -d "$driver_store"
[[ ! -e $scratch/ambiguous/$firmware_path/duplicate.mbn ]]
printf 'one' >"$driver_store/matching/duplicate.mbn"
run_extractor --stage "$scratch/identical" -d "$driver_store"
[[ $(<"$scratch/identical/$firmware_path/duplicate.mbn") == one ]]
echo "ok - differing variants require a unique match"
# A name that matches another as a regular expression keeps its manifest line.
printf 'first' >"$driver_store/matching/dotXmbn"
printf 'second' >"$driver_store/matching/dot.mbn"
printf '%s\0' "$firmware_path/dotXmbn" >"$node/firmware-name"
run_extractor --install --no-rebuild -d "$driver_store"
printf '%s\0' "$firmware_path/dot.mbn" >"$node/firmware-name"
run_extractor --install --no-rebuild -d "$driver_store"
[[ $(grep -c /dot "$scratch/root/var/lib/omarchy/qcom-firmware/manifest") == 2 ]]
echo "ok - manifest updates match firmware names exactly"
# A packaged zap shader still needs an initramfs entry when nothing is missing.
printf '%s\0' "$firmware_path/qccdsp8380.mbn" >"$node/firmware-name"
mkdir -p "$dt_root/gpu@0/zap-shader"
printf '%s\0' "$firmware_path/qcdxkmsuc8380.mbn" >"$dt_root/gpu@0/zap-shader/firmware-name"
zap="$firmware_path/qcdxkmsuc8380.mbn"
config="$scratch/root/etc/mkinitcpio.conf.d/qcom-firmware.conf"
for compression in zstd xz; do
if [[ $compression == zstd ]]; then suffix=zst; else suffix=xz; fi
printf 'zap' | "$compression" -c >"$firmware_root/$zap.$suffix"
run_extractor --install --no-rebuild -d "$driver_store"
[[ -z $(run_extractor --list-missing) ]]
grep -Fq "$zap.$suffix" "$config"
rm "$firmware_root/$zap.$suffix"
done
echo "ok - zstd and xz firmware are loadable and included in the initramfs"
# Arch Linux ARM's kernel loads xz firmware but not zstd.
set_kernel_config CONFIG_FW_LOADER_COMPRESS_XZ=y
printf 'zap' | zstd -c >"$firmware_root/$zap.zst"
[[ $(run_extractor --list-missing) == "$zap" ]]
rm "$firmware_root/$zap.zst"
set_kernel_config CONFIG_FW_LOADER_COMPRESS_ZSTD=y CONFIG_FW_LOADER_COMPRESS_XZ=y
echo "ok - zstd firmware counts as missing when the kernel cannot load it"
printf 'zap' | gzip >"$firmware_root/$zap.gz"
[[ $(run_extractor --list-missing) == "$zap" ]]
run_extractor --install --no-rebuild -d "$driver_store"
[[ ! -e $config ]]
printf 'zap' >"$driver_store/matching/qcdxkmsuc8380.mbn"
run_extractor --install --no-rebuild -d "$driver_store"
cmp "$driver_store/matching/qcdxkmsuc8380.mbn" "$firmware_root/updates/$zap"
grep -Fq "updates/$zap" "$config"
echo "ok - gzip does not hide missing firmware or prevent extraction"
# Compressed updates must not take precedence over a plain packaged file.
mv "$firmware_root/updates/$zap" "$firmware_root/$zap"
printf 'update' | zstd -c >"$firmware_root/updates/$zap.zst"
run_extractor --install --no-rebuild -d "$driver_store"
grep -Fq "$firmware_root/$zap" "$config"
if grep -Fq "updates/$zap" "$config"; then
echo "not ok - firmware selection differs from the kernel search order" >&2
exit 1
fi
printf 'update' >"$firmware_root/updates/$zap"
run_extractor --install --no-rebuild -d "$driver_store"
grep -Fq "updates/$zap" "$config"
echo "ok - plain firmware is preferred before compressed directory overrides"
limine-update() { printf 'rebuild\n' >>"$QCOM_FW_ROOT/rebuilds"; }
export -f limine-update
run_extractor --install -d "$driver_store"
[[ ! -e $scratch/root/rebuilds ]]
rm "$scratch/root/etc/mkinitcpio.conf.d/qcom-firmware.conf"
run_extractor --install -d "$driver_store"
[[ $(<"$scratch/root/rebuilds") == rebuild ]]
echo "ok - configuration-only changes rebuild the initramfs once"
# Reruns: an earlier stage, even an empty one, must not hide a later source.
rerun="$scratch/rerun"
rerun_dt="$rerun/device-tree"
adsp="qcom/glymur/vendor/board/qcadsp.mbn"
dtb="qcom/glymur/vendor/board/adsp_dtbs.elf"
windows_store="$rerun/windows/nvme0n1p3/Windows/System32/DriverStore/FileRepository/adsp.inf_1"
mkdir -p "$rerun_dt/remoteproc@0" "$rerun/root/run" "$rerun/empty-store" \
"$rerun/store" "$rerun/adsp-only-store" "$windows_store"
printf '%s\0%s\0' "$adsp" "$dtb" >"$rerun_dt/remoteproc@0/firmware-name"
printf 'store-adsp' >"$rerun/store/qcadsp.mbn"
printf 'store-dtb' >"$rerun/store/adsp_dtbs.elf"
printf 'first-adsp' >"$rerun/adsp-only-store/qcadsp.mbn"
printf 'windows-adsp' >"$windows_store/qcadsp.mbn"
printf 'windows-dtb' >"$windows_store/adsp_dtbs.elf"
# For a subshell: NTFS partitions holding fake Windows trees $1/<device name>,
# by default one internal partition, or those that $TEST_PARTITIONS lists as
# "PATH|FSTYPE|TRAN|RM|HOTPLUG" lines, where a field may be empty. lsblk prints
# the requested columns as raw output does, an empty one as an empty string.
# The disk listing and each mount are recorded in $2.
# shellcheck disable=SC2329 # The stubs are exported to the extractor.
stub_windows_partition() {
lsblk() {
local arg previous="" path fstype tran rm hotplug column row
local -a columns=()
printf 'lsblk\n' >>"$TEST_SCAN_LOG"
[[ -z ${TEST_NO_WINDOWS-} ]] || return 0
for arg; do
if [[ $previous == -*o ]]; then IFS=, read -ra columns <<<"$arg"; fi
previous=$arg
done
while IFS='|' read -r path fstype tran rm hotplug; do
row=""
for column in "${columns[@]}"; do
case $column in
PATH) row+=" $path" ;;
FSTYPE) row+=" $fstype" ;;
TRAN) row+=" $tran" ;;
RM) row+=" $rm" ;;
HOTPLUG) row+=" $hotplug" ;;
esac
done
printf '%s\n' "${row# }"
done <<<"${TEST_PARTITIONS:-/dev/nvme0n1p3|ntfs|nvme|0|0}"
}
mount() {
local device=${*: -2:1} mount_point=${*: -1}
printf 'mount %s\n' "$device" >>"$TEST_SCAN_LOG"
cp -R "$TEST_WINDOWS/${device##*/}/." "$mount_point/"
}
umount() { find "$1" -mindepth 1 -delete; }
export -f lsblk mount umount
export TEST_WINDOWS=$1 TEST_SCAN_LOG=$2
}
(
stub_windows_partition "$rerun/windows" "$rerun/scan.log"
run_rerun() {
QCOM_FW_DT_ROOT="$rerun_dt" \
QCOM_FW_FIRMWARE_ROOT="$rerun/firmware" \
QCOM_FW_ROOT="$rerun/root" \
QCOM_FW_TEST_EUID=${QCOM_FW_TEST_EUID-0} \
PATH="$test_bin:$PATH" \
bash "$extractor" "$@"
}
reset_installed() { rm -rf "$rerun/firmware" "$rerun/root/var" "$TEST_SCAN_LOG"; mkdir -p "$rerun/firmware"; }
installed() { cat "$rerun/firmware/updates/$1"; }
stage="$rerun/stage"
run_rerun --stage "$stage" -d "$rerun/empty-store"
[[ -f $stage/manifest && ! -s $stage/manifest ]]
run_rerun --stage "$stage" -d "$rerun/store"
[[ $(<"$stage/$adsp") == store-adsp && $(<"$stage/$dtb") == store-dtb ]]
[[ $(grep -c . "$stage/manifest") == 2 ]]
output=$(run_rerun --stage "$stage")
[[ $output == *"already staged"* ]]
[[ ! -e $TEST_SCAN_LOG ]]
echo "ok - a stage rerun adds files an earlier, empty stage lacked"
partial="$rerun/partial-stage"
run_rerun --stage "$partial" -d "$rerun/adsp-only-store"
run_rerun --stage "$partial" -d "$rerun/store"
[[ $(<"$partial/$adsp") == first-adsp && $(<"$partial/$dtb") == store-dtb ]]
[[ $(grep -c "^$adsp " "$partial/manifest") == 1 && $(grep -c . "$partial/manifest") == 2 ]]
echo "ok - a stage rerun keeps staged files and their manifest lines"
# Only the test switch stands in for root, not the test path prefix.
rm -f "$TEST_SCAN_LOG"
if output=$(QCOM_FW_TEST_EUID=1000 run_rerun --stage "$rerun/user-stage" 2>&1); then
echo "not ok - a first stage without root read the partitions" >&2
exit 1
fi
[[ $output == *"must run as root to read the Windows partitions"* && ! -e $TEST_SCAN_LOG ]]
if QCOM_FW_TEST_EUID=1000 run_rerun --install --no-rebuild -d "$rerun/store" >/dev/null 2>&1; then
echo "not ok - an install without root went ahead" >&2
exit 1
fi
echo "ok - reading the partitions and installing need root"
# A rerun without root cannot read Windows, so it keeps the stage it has.
user_partial="$rerun/user-partial"
run_rerun --stage "$user_partial" -d "$rerun/adsp-only-store"
output=$(QCOM_FW_TEST_EUID=1000 run_rerun --stage "$user_partial")
[[ $output == *"already staged in $user_partial; run as root"* && ! -e $TEST_SCAN_LOG ]]
[[ $(grep -c . "$user_partial/manifest") == 1 ]]
echo "ok - a stage rerun without root keeps a partial stage and succeeds"
# Neither a stage rerun nor an install reads or rewrites the stage manifest
# once per firmware name. External commands naming it are counted; reads
# with bash builtins are not.
probe="$rerun/probe-stage"
run_rerun --stage "$probe" -d "$rerun/adsp-only-store"
# shellcheck disable=SC2329 # The wrappers are exported to the extractor.
(
manifest_access() {
local tool=$1 arg
shift
for arg; do
if [[ $arg == "$probe/manifest"* ]]; then
printf '%s\n' "$tool" >>"$rerun/manifest.log"
break
fi
done
command "$tool" "$@"
}
awk() { manifest_access awk "$@"; }
cat() { manifest_access cat "$@"; }
cp() { manifest_access cp "$@"; }
grep() { manifest_access grep "$@"; }
mv() { manifest_access mv "$@"; }
sed() { manifest_access sed "$@"; }
export -f manifest_access awk cat cp grep mv sed
export probe rerun
run_rerun --stage "$probe" -d "$rerun/store"
[[ $(<"$probe/$dtb") == store-dtb && $(command grep -c . "$rerun/manifest.log") -le 1 ]]
rm "$rerun/manifest.log"
reset_installed
run_rerun --install --no-rebuild --stage-dir "$probe"
[[ $(installed "$adsp") == first-adsp && $(installed "$dtb") == store-dtb ]]
[[ ! -e $rerun/manifest.log || $(command grep -c . "$rerun/manifest.log") -le 1 ]]
)
echo "ok - no per-name stage manifest scans or rewrites"
mkdir -p "$rerun/empty-stage"
: >"$rerun/empty-stage/manifest"
reset_installed
run_rerun --install --no-rebuild --stage-dir "$rerun/empty-stage"
[[ $(installed "$adsp") == windows-adsp && $(installed "$dtb") == windows-dtb ]]
echo "ok - an empty stage does not hide the Windows partition"
dtb_stage="$rerun/dtb-stage"
mkdir -p "$dtb_stage/${dtb%/*}"
printf 'staged-dtb' >"$dtb_stage/$dtb"
printf '%s %s %s\n' "$dtb" 0 test >"$dtb_stage/manifest"
reset_installed
run_rerun --install --no-rebuild --stage-dir "$dtb_stage"
[[ $(installed "$dtb") == staged-dtb && $(installed "$adsp") == windows-adsp ]]
echo "ok - Windows supplies what a partial stage lacks, and the stage comes first"
reset_installed
run_rerun --install --no-rebuild --stage-dir "$stage"
[[ $(installed "$adsp") == store-adsp && $(installed "$dtb") == store-dtb ]]
[[ ! -e $TEST_SCAN_LOG ]]
echo "ok - a complete stage is used without mounting Windows"
printf 'unlisted' >"$dtb_stage/$adsp"
reset_installed
run_rerun --install --no-rebuild --stage-dir "$dtb_stage"
[[ $(installed "$adsp") == windows-adsp ]]
echo "ok - only files recorded in the stage manifest count as staged"
# A manifest line whose file is gone does not count as staged either.
rm "$dtb_stage/$dtb"
reset_installed
run_rerun --install --no-rebuild --stage-dir "$dtb_stage"
[[ $(installed "$dtb") == windows-dtb ]]
printf 'staged-dtb' >"$dtb_stage/$dtb"
echo "ok - a manifest line without its file does not count as staged"
reset_installed
run_rerun --install --no-rebuild --stage-dir "$stage" -d "$rerun/adsp-only-store"
[[ $(installed "$adsp") == first-adsp && ! -e $rerun/firmware/updates/$dtb ]]
[[ ! -e $TEST_SCAN_LOG ]]
echo "ok - -d replaces the stage and the Windows partitions"
# After a full-disk install the stage is the only copy, even when a later
# device tree names a staged image under another path.
moved="$rerun/moved-stage"
old_adsp="qcom/glymur/vendor/old-board/qcadsp.mbn"
mkdir -p "$moved/${old_adsp%/*}" "$moved/${dtb%/*}"
printf 'moved-adsp' >"$moved/$old_adsp"
printf 'staged-dtb' >"$moved/$dtb"
printf '%s 0 test\n%s 0 test\n' "$old_adsp" "$dtb" >"$moved/manifest"
reset_installed
run_rerun --install --no-rebuild --stage-dir "$moved"
[[ $(installed "$adsp") == windows-adsp && $(installed "$dtb") == staged-dtb ]]
reset_installed
TEST_NO_WINDOWS=1 run_rerun --install --no-rebuild --stage-dir "$moved"
[[ -s $TEST_SCAN_LOG ]]
[[ $(installed "$adsp") == moved-adsp && $(installed "$dtb") == staged-dtb ]]
printf '%s 0 test\n' "$dtb" >"$moved/manifest"
reset_installed
TEST_NO_WINDOWS=1 run_rerun --install --no-rebuild --stage-dir "$moved"
[[ ! -e $rerun/firmware/updates/$adsp ]]
echo "ok - a staged file under an older device-tree path is used after Windows"
# The stage does not override Windows refusing ambiguous variants.
mkdir -p "${windows_store%_1}_2"
printf 'other-adsp' >"${windows_store%_1}_2/qcadsp.mbn"
printf '%s 0 test\n%s 0 test\n' "$old_adsp" "$dtb" >"$moved/manifest"
reset_installed
run_rerun --install --no-rebuild --stage-dir "$moved" 2>"$rerun/warnings"
grep -Fq "refusing ambiguous qcadsp.mbn" "$rerun/warnings"
[[ ! -e $rerun/firmware/updates/$adsp && $(installed "$dtb") == staged-dtb ]]
rm -r "${windows_store%_1}_2"
echo "ok - a staged file by name does not replace a refused ambiguous match"
# An installed system reads only internal disks; --stage, in the live
# session, also reads removable and external ones. The disks: a USB stick,
# a removable card that is neither USB nor hotplug, an internal disk whose
# TRAN lsblk leaves empty, and a hotplug NVMe disk.
(
export TEST_WINDOWS="$rerun/disks"
export TEST_PARTITIONS=$'/dev/sda1|ntfs|usb|1|0\n/dev/mmcblk0p1|ntfs|mmc|1|0\n/dev/nvme0n1p3|ntfs||0|0\n/dev/nvme1n1p3|ntfs|nvme|0|1'
external=(sda1 mmcblk0p1 nvme1n1p3)
inf="Windows/System32/DriverStore/FileRepository/adsp.inf_1"
mkdir -p "$TEST_WINDOWS"/{sda1,mmcblk0p1,nvme0n1p3,nvme1n1p3}/"$inf"
printf 'internal-adsp' >"$TEST_WINDOWS/nvme0n1p3/$inf/qcadsp.mbn"
for disk in "${external[@]}"; do
printf 'external-dtb' >"$TEST_WINDOWS/$disk/$inf/adsp_dtbs.elf"
done
reset_installed
output=$(run_rerun --install --no-rebuild)
[[ $(installed "$adsp") == internal-adsp && ! -e $rerun/firmware/updates/$dtb ]]
[[ $output == *"not reading removable or external disk partition(s): /dev/sda1 /dev/mmcblk0p1 /dev/nvme1n1p3;"* ]]
for disk in "${external[@]}"; do
if grep -qx "mount /dev/$disk" "$TEST_SCAN_LOG"; then
echo "not ok - an installed system mounted the removable or external /dev/$disk" >&2
exit 1
fi
done
reset_installed
run_rerun --stage "$rerun/all-disks"
[[ $(<"$rerun/all-disks/$adsp") == internal-adsp && $(<"$rerun/all-disks/$dtb") == external-dtb ]]
for disk in nvme0n1p3 "${external[@]}"; do
grep -qx "mount /dev/$disk" "$TEST_SCAN_LOG"
done
)
echo "ok - an installed system reads only internal disks, the live session all"
)
# A board package can list firmware names that must not come from Windows.
board="$scratch/board"
board_dt="$board/device-tree"
provided_dir="$board/provided.d"
router="qcom/glymur/vendor/board/usb4-router.bin"
mkdir -p "$board_dt/remoteproc@0" "$board_dt/usb4@0" "$board/store" "$board/root" "$provided_dir"
printf 'vendor,board\0qcom,glymur\0' >"$board_dt/compatible"
printf '%s\0%s\0' "$adsp" "$dtb" >"$board_dt/remoteproc@0/firmware-name"
printf '%s\0' "$router" >"$board_dt/usb4@0/firmware-name"
printf 'store-adsp' >"$board/store/qcadsp.mbn"
printf 'store-dtb' >"$board/store/adsp_dtbs.elf"
printf 'store-router' >"$board/store/usb4-router.bin"
# Runs the extractor through any command in $board_wrapper.
board_wrapper=()
run_board() {
QCOM_FW_DT_ROOT="$board_dt" \
QCOM_FW_FIRMWARE_ROOT="$board/firmware" \
QCOM_FW_ROOT="$board/root" \
QCOM_FW_TEST_EUID=0 \
QCOM_FW_PROVIDED_DIR="$provided_dir" \
PATH="$test_bin:$PATH" \
"${board_wrapper[@]}" bash "$extractor" "$@"
}
expected_missing=$(printf '%s\n' "$adsp" "$dtb")
[[ $(run_board --list-missing) == "$expected_missing"$'\n'"$router" ]]
printf '# Made by the board package from the Windows driver.\n %s # router image\n\n' "$router" \
>"$provided_dir/other,board.list"
[[ $(run_board --list-missing) == "$expected_missing"$'\n'"$router" ]]
# A compatible string names a list in provided.d, never one in a subdirectory.
cp "$board_dt/compatible" "$board/compatible.saved"
printf 'vendor/board\0' >"$board_dt/compatible"
mkdir -p "$provided_dir/vendor"
printf '%s\n' "$router" >"$provided_dir/vendor/board.list"
[[ $(run_board --list-missing) == "$expected_missing"$'\n'"$router" ]]
mv "$board/compatible.saved" "$board_dt/compatible"
rm -r "$provided_dir/vendor"
echo "ok - firmware listed for another board, or under a path, is still reported missing"
mv "$provided_dir/other,board.list" "$provided_dir/vendor,board.list"
[[ $(run_board --list-missing) == "$expected_missing" ]]
run_board --stage "$board/stage" -d "$board/store"
[[ -f $board/stage/$adsp && ! -e $board/stage/$router ]]
[[ $(grep -c . "$board/stage/manifest") == 2 ]]
run_board --install --no-rebuild -d "$board/store"
[[ $(<"$board/firmware/updates/$adsp") == store-adsp && ! -e $board/firmware/updates/$router ]]
if grep -Fq "$router" "$board/root/var/lib/omarchy/qcom-firmware/manifest"; then
echo "not ok - provided firmware is recorded as extracted" >&2
exit 1
fi
[[ -z $(run_board --list-missing) ]]
echo "ok - listed firmware is neither searched for, installed nor reported missing"
# List syntax: a '#' is a comment only at the start of a line or after a space,
# and an entry with a space inside is skipped with a warning, not joined up.
board_list="$provided_dir/vendor,board.list"
cp "$board_list" "$board/list.saved"
printf '%s\n' "${router%/*}/usb4- router.bin" "$router#2" >"$board_list"
[[ $(run_board --list-missing 2>"$board/warnings") == "$router" ]]
grep -Fq "ignoring '${router%/*}/usb4- router.bin' in $board_list" "$board/warnings"
printf '\t%s\t# router image\r\n' "$router" >"$board_list"
[[ -z $(run_board --list-missing) ]]
echo "ok - list comments need a space before '#', and names with spaces are skipped"
# Root reads any file, so as root (CI runs these tests in a root container)
# run without the capabilities that let it.
unreadable_skip=""
if ((EUID == 0)); then
board_wrapper=(setpriv '--bounding-set=-dac_override,-dac_read_search'
'--inh-caps=-dac_override,-dac_read_search' --)
if ! "${board_wrapper[@]}" true 2>/dev/null; then
unreadable_skip="root can read it, and setpriv could not drop that capability"
fi
fi
chmod 000 "$board_list"
if [[ -z $unreadable_skip ]] && "${board_wrapper[@]}" cat "$board_list" >/dev/null 2>&1; then
unreadable_skip="this user can still read a mode 000 file"
fi
if [[ -z $unreadable_skip ]]; then
output=$(run_board --list-missing 2>"$board/warnings")
fi
board_wrapper=()
chmod 644 "$board_list"
if [[ -z $unreadable_skip ]]; then
[[ $output == "$router" ]]
grep -Fq "ignoring $board_list: it cannot be read" "$board/warnings"
echo "ok - an unreadable list is skipped with a warning"
else
echo "ok - an unreadable list is skipped with a warning # SKIP $unreadable_skip"
fi
cp "$board/list.saved" "$board_list"
# A copy an earlier run installed before the name was listed is left in place,
# with a note, and no longer recorded. It is never removed, even with the same
# bytes, because a board's own tool may have written that file.
board_manifest="$board/root/var/lib/omarchy/qcom-firmware/manifest"
board_updates="$board/firmware/updates"
install_unlisted_router() {
rm -f "$board_updates/$router"
mv "$board_list" "$board/list.off"
run_board --install --no-rebuild -d "$board/store" >/dev/null
mv "$board/list.off" "$board_list"
[[ $(<"$board_updates/$router") == store-router ]]
grep -q "^$router " "$board_manifest"
}
install_listed_router() {
local output
output=$(run_board --install --no-rebuild -d "$board/store")
[[ $(<"$board_updates/$router") == store-router ]]
[[ $output == *"note: an earlier run installed $board_updates/$router, but $router is now listed in $board_list; the file is left in place and no longer recorded"* ]]
if grep -q "^$router " "$board_manifest"; then
echo "not ok - a listed name is still recorded as installed" >&2
exit 1
fi
}
install_unlisted_router
install_listed_router
[[ $(<"$board_updates/$adsp") == store-adsp ]]
install_unlisted_router
# The board's tool writes the same bytes there as a new file.
rm "$board_updates/$router"
printf 'store-router' >"$board_updates/$router"
install_listed_router
rm "$board_updates/$router"
echo "ok - a listed name's earlier copy is left in place and no longer recorded"
# A board also lists names nothing in Windows should supply: SOCCP firmware
# the machine's boot firmware loads before Linux, which no driver store holds,
# and a CDSP image the board holds back although Windows and the stage have it.
soccp="qcom/glymur/vendor/board/soccp.mbn"
soccp_dtb="qcom/glymur/vendor/board/soccp_dtb.mbn"
cdsp="qcom/glymur/vendor/board/qccdsp.mbn"
cdsp_dtb="qcom/glymur/vendor/board/cdsp_dtbs.elf"
board_store="$board/windows/nvme0n1p3/Windows/System32/DriverStore/FileRepository"
mkdir -p "$board_dt/remoteproc@1" "$board_dt/remoteproc@2" "$board/root/run" \
"$board_store/cdsp.inf_1" "$board_store/adsp.inf_1"
printf '%s\0%s\0' "$soccp" "$soccp_dtb" >"$board_dt/remoteproc@1/firmware-name"
printf '%s\0%s\0' "$cdsp" "$cdsp_dtb" >"$board_dt/remoteproc@2/firmware-name"
printf 'windows-cdsp' >"$board_store/cdsp.inf_1/qccdsp.mbn"
printf 'windows-cdsp-dtb' >"$board_store/cdsp.inf_1/cdsp_dtbs.elf"
printf 'windows-adsp' >"$board_store/adsp.inf_1/qcadsp.mbn"
printf '%s\n' "# Loaded by the boot firmware." "$soccp" "$soccp_dtb" \
"# Not tested on this board." "$cdsp" "$cdsp_dtb" >>"$board_list"
cdsp_stage="$board/cdsp-stage"
mkdir -p "$cdsp_stage/${cdsp%/*}"
printf 'staged-cdsp' >"$cdsp_stage/$cdsp"
printf 'staged-cdsp-dtb' >"$cdsp_stage/$cdsp_dtb"
printf '%s 0 test\n%s 0 test\n' "$cdsp" "$cdsp_dtb" >"$cdsp_stage/manifest"
(
stub_windows_partition "$board/windows" "$board/scan.log"
[[ -z $(run_board --list-missing) ]]
output=$(run_board --stage "$board/later-stage")
[[ $output == *"already installed or listed for this board"* && ! -e $board/later-stage ]]
output=$(run_board --install --no-rebuild)
[[ $output == *"skipping $soccp: listed in $board_list"* ]]
run_board --install --no-rebuild --stage-dir "$cdsp_stage"
[[ ! -e $TEST_SCAN_LOG ]]
[[ ! -e $board_updates/$soccp && ! -e $board_updates/$cdsp && ! -e $board_updates/$cdsp_dtb ]]
echo "ok - with only listed names left, nothing reads the disks or installs them"
# Control: without the list, the same machine looks for them on the disks.
mv "$board_list" "$board/list.off"
[[ $(run_board --list-missing | grep -c .) == 5 ]]
run_board --install --no-rebuild --stage-dir "$cdsp_stage"
grep -qx lsblk "$TEST_SCAN_LOG"
grep -qx 'mount /dev/nvme0n1p3' "$TEST_SCAN_LOG"
[[ $(<"$board_updates/$cdsp") == staged-cdsp ]]
echo "ok - without the list, the same machine scans the disks for them (control)"
# Listing them again, without a scan, stops recording the copies that run
# installed and leaves them in place with a note.
mv "$board/list.off" "$board_list"
rm "$TEST_SCAN_LOG"
output=$(run_board --install --no-rebuild --stage-dir "$cdsp_stage")
[[ ! -e $TEST_SCAN_LOG ]]
[[ $(<"$board_updates/$cdsp") == staged-cdsp && $(<"$board_updates/$cdsp_dtb") == staged-cdsp-dtb ]]
[[ $output == *"note: an earlier run installed $board_updates/$cdsp,"* ]]
[[ $output == *"note: an earlier run installed $board_updates/$cdsp_dtb,"* ]]
if grep -qe "^$cdsp " -e "^$cdsp_dtb " "$board_manifest"; then
echo "not ok - listed names are still recorded as installed" >&2
exit 1
fi
# The board package or the user removes them.
rm "$board_updates/$cdsp" "$board_updates/$cdsp_dtb"
echo "ok - listing the names again leaves the copies that run installed, unrecorded"
# Something else missing still sends the extractor to Windows, for it alone.
rm "$board_updates/$adsp"
run_board --install --no-rebuild --stage-dir "$cdsp_stage"
grep -qx lsblk "$TEST_SCAN_LOG"
[[ $(<"$board_updates/$adsp") == windows-adsp ]]
[[ ! -e $board_updates/$soccp && ! -e $board_updates/$cdsp && ! -e $board_updates/$cdsp_dtb ]]
echo "ok - while Windows is read for other firmware, listed names are not installed"
)