omarchy-pkgs/docs/repository-host.md

The repository host (abandoned)

This pipeline is abandoned. Packages are built and published by GitHub workflows: see the README. Don't follow this page to publish anything.

The package pipeline used to run on one machine that built, signed and published every channel on a timer. This page is kept as the reference for the tooling that was written for it and is still in the tree: bin/repo, the units in systemd/, and the host side of the release train (releases.md).

That tooling writes to the same bucket as CI. If you run any of it (release, sync, promote, push, deploy, advance, remove), these rules apply:

• Do not run bin/repo release, sync, push or deploy for a package CI publishes. Merge a PR, or dispatch publish.yml, instead.
• Keep the omarchy-auto-release-* and omarchy-check-versions timers disabled while CI publishes the same channels. Two writers on one channel database overwrite each other.
• Never pass --prune to bin/repo sync. The host's tree does not hold what CI published, and --prune deletes remote files the local tree lacks.
• Never rebuild a version that is already published. Bump pkgrel.

Setting up a host

Abandoned. Kept as reference for the tooling; don't follow it to publish.

aarch64 under emulation

The repository host builds every architecture it publishes on the same machine. A foreign architecture runs under QEMU user emulation, which bin/build checks by actually running a container for the target platform. Rootful Docker registers QEMU on first use. Rootless Podman uses the host's registration and prints the one-time Arch setup commands when it is missing or lacks the credential flag required by sudo inside the builder:

# Verify
podman run --rm --platform linux/arm64 docker.io/library/alpine:latest uname -m
# Should output: aarch64

Note: emulated builds are much slower than native ones.

Published architectures

Abandoned. Kept as reference for the tooling; don't follow it to publish.

helpers/paths.sh names the architectures the host's scheduled pipeline and release train work on. CI does not read it:

PUBLISHED_ARCHES="${OMARCHY_ARCHES:-x86_64}"

That list drives the whole scheduled pipeline. check-versions compares PKGBUILDs against each architecture's channel databases and writes one queue file per channel and architecture (.sync-needed-<channel>-<arch>); auto-release <channel> works through the queues one architecture at a time, each with its own backoff (.build-failed-<channel>-<arch>), so a failing build on one architecture never holds up the other; and the release train advances channels with --arch all: it takes one host-wide lock and verifies every architecture's source database before moving any of them. The first entry is the reference architecture the release train observes channels through. Rerunning the same advance after a failed remote sync completes it safely.

Adding an architecture to the scheduled pipeline is therefore one checked-in change to that list: the next check-versions tick queues everything the new architecture lacks, and the next auto-release tick starts building it. CI does not read this list: it builds the architectures in CI_ARCHES, both by default. For a one-off run, override it directly:

OMARCHY_ARCHES=x86_64 bin/check-versions
OMARCHY_ARCHES=aarch64 bin/check-versions
OMARCHY_ARCHES="x86_64 aarch64" bin/check-versions

The builder image bootstraps omarchy-keyring from the x86_64 tree for every architecture, so the first build of a new architecture does not depend on a repository that only it can create.

Release in one command

Abandoned. Kept as reference for the tooling; don't follow it to publish.

The release command is smart and incremental - it only builds packages that have changed or are missing. You generally don't need to specify a package manually unless you are debugging a specific failure.

When a package fails, a completed build run still signs and publishes the packages that succeeded. Failed packages and their blocked dependents remain queued with failure backoff; retries compare against the updated repository and skip the published versions. Only artifacts recorded by fully completed package builds are eligible for a partial release. An interrupted build, a failed publication step, or an incomplete pair using deferred runtime dependencies still stops the release. Reports distinguish partial publication from complete success.

# Build changed/new packages, sign, promote, clean, update, and sync
bin/repo release

# Stable Mirror
bin/repo release --mirror stable

# ARM64
bin/repo release --arch aarch64

# Build one package (still skipped when its version is already published)
bin/repo release --package omarchy-nvim

# Show what would build without signing/promoting/syncing
bin/repo release --dry-run

Step-by-Step

bin/repo build                          # Build (unsigned)
bin/repo sign                           # Sign packages
bin/repo promote                        # Copy to production
bin/repo clean                          # Remove old versions
bin/repo update                         # Update database
bin/repo sync                           # Sync to remote

Building Heavy Packages Locally

Abandoned. Kept as reference for the tooling; don't follow it to publish.

Large packages build faster on a local machine than on the server. Build them here, then hand the artifacts to the repository host, which signs and publishes them:

bin/repo deploy --package nvidia-580xx-utils   # Build here, publish from the host

deploy is build followed by push. The two steps are also available separately when a build needs inspecting before it ships:

bin/build --package nvidia-580xx-utils         # Build on the fast machine
bin/repo push --package nvidia-580xx-utils     # Upload + publish on the host

push uploads to the host's build-output/, verifies checksums, and runs bin/upload-prebuilt there. Do not publish from a local checkout instead: a local machine holds neither the complete repository nor a signing key.

Name the package. build plans against the local repository database, or against the public channel's when there is none. deploy refuses to run unscoped when the local database is missing. Unscoped builds belong on the repository host, where bin/repo release does the same job against a real database.

Every other bin/repo command — sign, promote, update, clean, advance, remove, sync, release — works on the published tree. Run them from anywhere: with a repository host configured bin/repo forwards them over ssh to run there (see Build trigger and the build host), and --local forces execution on the current machine. bin/repo build forwards too; call bin/build directly, or pass --local, to build on this machine. push and deploy always run locally.

Commands

Abandoned. Kept as reference for the tooling; don't follow it to publish.

Global Flags

These flags can be used with all commands:

• --mirror <edge|rc|stable>: Selects the channel (default: edge).
• --local: Run on this machine instead of forwarding to the repository host.
• --arch <x86_64|aarch64>: Selects the target architecture (default: x86_64).

Build

bin/repo build                                   # All packages (x86_64, edge)
bin/repo build --arch aarch64                    # ARM64
bin/repo build --mirror rc                       # Release-candidate channel
bin/repo build --mirror stable                   # Stable channel
bin/repo build --package yay cursor-bin          # Specific packages
bin/repo build --dry-run                         # Show what would build

Output: Unsigned .pkg.tar.zst in build-output/. Only builds packages whose version differs from what the repository holds.

Use --dry-run to show the build plan without running makepkg.

Sign

bin/repo sign

Reads the GPG key and passphrase from the environment and signs all packages in build-output/. Supports --arch and --mirror.

Promote

bin/repo promote                    # Copy to production
bin/repo promote --arch aarch64     # ARM64
bin/repo promote --dry-run          # Preview

Copies signed packages from build-output/ → pkgs.omarchy.org/.

Clean

bin/repo clean                      # Keep 2 versions
bin/repo clean --keep 3             # Keep 3 versions
bin/repo clean --dry-run            # Preview

Removes old package versions from the file system. Does not update the database.

Update

bin/repo update                     # Update database

Updates the repository database (adding the newest version of each package). Run this after promote or clean.

Sync Repository

bin/repo sync                           # Sync current arch/mirror
bin/repo sync --mirror stable           # Sync stable
bin/repo sync --arch aarch64            # Sync ARM64
bin/repo sync --skip-prod-check         # No confirmation

Syncs package repositories to the remote server using rclone based on the configured mirror and architecture.

Uploads are additive. A local tree is not authoritative about what belongs on the remote — pkgs.omarchy.org/ is gitignored, and packages built on another machine exist only there — so sync never deletes by default. To take a package out of the channels, use the unpublish.yml workflow.

For the same reason sync refuses to publish a repository database built from a tree holding fewer packages than the remote database already lists. The database is what pacman resolves against, so a partial one hides every package it does not know about even though the files are still on the mirror. Use bin/repo push to publish packages built on another machine.

Deploy

bin/repo deploy --package nvidia-580xx-utils   # Build locally, publish from the host
bin/repo deploy --host root@example.com        # Point at a specific repo host
bin/repo deploy --dry-run                      # Show the plan, change nothing

Runs build then push in one command. The repository host is resolved before the build starts, so a missing --host fails immediately rather than after a long compile.

Push to the Repository Host

bin/repo push                                  # Push everything in build-output
bin/repo push --package nvidia-580xx-utils     # Push one package
bin/repo push --mirror stable --arch aarch64   # Pick mirror and architecture
bin/repo push --host root@example.com          # Override the repo host
bin/repo push --dry-run                        # Show the plan, transfer nothing

Uploads packages from build-output/ to the repository host and publishes them there with bin/upload-prebuilt (sign → promote → update → sync). Use it when a package is quicker to build on a local machine than on the server.

Publishing happens on the host rather than locally for two reasons: the host has a GPG signing key (CI has its own copy in the publish environment), and it holds the repository tree that its database and sync are built from. Local machines therefore need no secrets.

The host comes from --host, $OMARCHY_REPO_HOST, then .repo-host. The setting is named for the repository rather than for building, which happens wherever you like. The same setting tells bin/omarchy-pkgs release which host to poke after a release push.

--package means the same thing as it does to build: a pkgbase, whose every output ships together. Pushing nvidia-580xx-utils carries nvidia-580xx-dkms and opencl-nvidia-580xx with it, because that is what the build produced. An output's own name still selects just that one, for publishing a single package on purpose. Omit --package to push everything built.

Publishing signs and promotes everything staged on the host, not just what this push uploaded, so push stops when it finds packages already staged there — usually leftovers from a failed run. Remove them on the host, or pass --include-staged to publish them too.

Other host commands

bin/repo advance --from rc --to stable            # Ship the tested rc channel to stable
bin/repo advance --from edge --to rc              # Open a train: carry edge forward into rc
bin/repo advance --from edge --to rc --package x  # Deliberate single-package advance
bin/repo advance --from rc --to stable --dry-run  # Preview without changing files
bin/repo bootstrap-rc                             # One-time initial rc seed from stable
bin/repo timers                      # Timer schedule, last runs, queues, backoff, lock
bin/repo list                        # List package metadata
bin/repo deploy                      # Build locally, then publish from the host
bin/repo push                        # Upload local builds to the host and publish
bin/repo remove <package>            # Remove package (host workflow; CI uses unpublish.yml)
bin/clean-docker                     # Clear Docker images/cache (forces fresh rebuild)

Build trigger and the build host

Abandoned. Kept as reference for the tooling; don't follow it to publish.

Release commands run from anywhere. bin/repo is a remote control: with a repository host configured, every command that operates on the published tree (release, build, sign, promote, update, clean, advance, bootstrap-rc, remove, sync, migrate) executes ON the host over ssh — the exact same code, run where the tree lives, so ssh'ing in and running the same commands by hand behaves identically. Pass --local to force execution on the current machine. list, push, deploy, and setup never forward (push/deploy exist precisely to move local builds to the host).

Without a configured host, commands run locally — which on the build host itself (no .repo-host there) is exactly right, and elsewhere the exact commands to run are printed by the orchestrator, with the 5-minute auto-release timer as the backstop.

The host setting is any destination ssh accepts, resolved in this order:

1. --host <dest>, for push, deploy and bin/omarchy-release
2. OMARCHY_REPO_HOST environment variable
3. the git-ignored .repo-host file (one line; # comments allowed)

The server layout lives under /root, so the value is root@<ip>, root@<hostname>, or — nicest — a Host alias from ~/.ssh/config that carries the user, key, and port:

# .repo-host
root@pkgs.example.com

All connections are plain ssh; nothing else is used to reach the host.

How "this machine is the build host" is detected — and its one caveat. The marker is the published database living in this checkout (pkgs.omarchy.org/<channel>/<arch>/omarchy.db*), the same convention the rest of bin/ uses. A workstation that once ran a full local bin/repo release carries that marker too and would infer it is the host. An explicitly configured destination always outranks the inference — --host, OMARCHY_REPO_HOST, and .repo-host are checked first, and only when none is set does local detection apply. So if a machine ever misidentifies itself, writing the real host into .repo-host is the fix.

Directory Structure

Abandoned. Kept as reference for the tooling; don't follow it to publish.

omarchy-pkgs/
├── pkgbuilds/                  # Source PKGBUILDs
│   └── package-name/
│       ├── PKGBUILD
│       └── .omarchy/
│           ├── package.json    # Source/sync/release metadata
│           └── upstream.sh     # Optional custom vendor release feed hook
├── build/
├── build-output/               # Unsigned packages (temporary)
│   ├── edge/                   # (rc/ and stable/ alongside, each x86_64 + aarch64)
│   ├── rc/
│   └── stable/
├── pkgs.omarchy.org/           # Signed packages (production)
│   ├── .release.lock           # Host-wide lock: one channel mutation at a time
│   ├── edge/                   # Each channel: x86_64/ and aarch64/
│   ├── rc/
│   └── stable/
└── bin/                        # CLI tools (on host)

Architecture-Specific Notes

Abandoned. Kept as reference for the tooling; don't follow it to publish.

x86_64

• Native builds (fast)
• Mirrors: mirror.omarchy.org, rackspace, pkgbuild.com

aarch64

• Built on the repository host like x86_64; under QEMU when the host is x86_64
• On an ARM host, package builds and the signing/database utility containers run natively; only an explicitly requested x86_64 package build is emulated
• Uses Arch Linux ARM repositories through the same HTTPS mirror for every channel (Arch Linux ARM publishes no dated snapshots to pin a channel's base)
• Additional repos: [alarm], [aur]
• Same workflow, just add --arch aarch64; the scheduled pipeline runs it automatically once aarch64 is in PUBLISHED_ARCHES
• Packages whose arch=() lacks aarch64 are skipped, not failed

Building for Both Architectures

# Build x86_64
bin/repo release --package myapp

# Build aarch64
bin/repo release --arch aarch64 --package myapp

# Sync both
bin/repo sync
bin/repo sync --arch aarch64

Version Management

Abandoned. Kept as reference for the tooling; don't follow it to publish.

Packages are only rebuilt if:

• PKGBUILD version differs from the repository version
• Package doesn't exist in production

A package that has to be rebuilt because something underneath it changed is handled by turning that into a version change: bin/sync-rebuilds bumps pkgrel when a dependency named in rebuild_on moves.

Scheduled releases on the host

Abandoned. Kept as reference for the tooling; don't follow it to publish.

Four systemd units in systemd/ drive unattended releases from the host.

All four units run every 5 minutes, staggered by a minute each, so a push reaches the mirror in minutes rather than hours:

1. check-versions (*:0/5): Pulls latest from git, compares PKGBUILD versions to published versions for every published architecture, creates one state file per channel and architecture if builds are needed
2. auto-release-edge (*:1/5): For each published architecture with a state file, builds all edge packages that need updates
3. auto-release-rc (*:2/5): Builds fast-ring packages for rc, from the main checkout like the other two — natively in the rc image, not copied from another channel. The pinned release pair is built separately by omarchy-release rc in the rc branch worktree
4. auto-release-stable (*:3/5): If a state file exists, builds release_ring=fast packages for stable

That cadence is only safe because of three guards:

• No overlap. Every channel-mutating run takes a host-wide lock (pkgs.omarchy.org/.release.lock). Scheduled runs take it non-blocking: if a build is already going, the tick exits immediately instead of queuing. Waiting would stack one stalled process per tick behind a long build and stampede when it finished. Manual commands still wait, as an operator expects. check-versions takes it too — its git pull would otherwise swap PKGBUILDs out from under a running build.
• Backoff on failure. A failed release records the attempt in .build-failed-<channel>-<arch> and backs off exponentially — 10m, 20m, 40m, up to a 6h ceiling — instead of rebuilding the same broken tree every 5 minutes. Any new commit clears the backoff immediately, since a push is the most likely fix. Clear it by hand with rm /root/.state/.build-failed-<channel>-<arch>.
• Quiet when idle. With nothing queued a tick exits without output, so the journal shows the runs that mattered rather than 288 no-ops a day.

bin/repo timers reports all of this: schedules, last results, what is queued, what is failing and when it will retry, and whether the lock is held.

Build reports

Abandoned. Kept as reference for the tooling; don't follow it to publish.

Every release run reports to Basecamp Campfire — not just failures, so a push can be followed all the way to the mirror without watching the host:

Event	Message
🔨 Started	channel, arch, host, the commit being built, and which packages are queued
✅ Published	the packages and versions that went out, duration, and the channel URL
📦 Promoted	what advance moved between channels (including the rc bootstrap)
📦 Nothing to publish	a queued run built nothing — see below
🔴 Failed	which step failed, the commit, and the last 25 log lines

Reports are tied to release runs, not timer ticks: a tick with nothing queued exits silently, so a quiet day is a quiet chat. That is what makes the "nothing to publish" report meaningful rather than routine — a release only runs when the version check queued work, so building nothing means the check and the builder disagree about what is out of date. It names the packages that were queued but not built, which is where to start looking.

By default everything posts to BASECAMP_CHATBOT_URL, the same chat the sync workflows use — one variable, nothing extra to configure.

If release traffic starts drowning that chat, give it its own: create a second Basecamp chat, add a chatbot integration to it, and export its lines URL alongside the existing one in /root/.omarchy/build-credentials:

export OMARCHY_RELEASE_CHATBOT_URL="https://3.basecamp.com/<account>/integrations/<key>/buckets/<project>/chats/<chat>/lines"

Release reports then go there while the sync workflows keep posting to BASECAMP_CHATBOT_URL. With neither set, reports are silently skipped. bin/setup reports which of the three cases applies.

Check on all of it with bin/repo timers — schedule, each unit's last run and whether it succeeded, what is queued, whether a release is running right now, and any failed units. Like the other host commands it forwards over ssh, so the build box's state is one command away from any machine:

bin/repo timers           # runs on the repository host when one is configured
bin/repo timers --local   # inspect this machine instead

State files are stored in /root/.state/:

• .sync-needed-<channel>-<arch> — the packages queued for that channel and architecture, one per line; the release run reads them to name what it is building
• .build-failed-<channel>-<arch> — consecutive failure count, timestamp, and the commit it failed on (drives the backoff; removing it forces a retry)

Legacy files without the architecture suffix are consumed once as x86_64 state, so upgrading the host does not lose an in-flight build.

Schedule (America/New_York)

Abandoned. Kept as reference for the tooling; don't follow it to publish.

Minute of every hour	Action
:00, :05, :10, …	check-versions (git pull + creates state files)
:01, :06, :11, …	auto-release-edge
:02, :07, :12, …	auto-release-rc
:03, :08, :13, …	auto-release-stable

Each release unit is a no-op when its channel has nothing queued, another run holds the lock, or the channel is in failure backoff.

Installation

Abandoned. Kept as reference for the tooling; don't follow it to publish.

ssh root@<host> 'cd /root/omarchy-pkgs && bin/setup --skip-timers'

bin/setup installs the dependencies, ensures Docker is running and creates the state directory. Without --skip-timers it also installs and enables the release timers, which must stay off while CI publishes. It works on Debian/Ubuntu and on Arch, and is idempotent, so run it again whenever a dependency is added.

The host does not need to be Arch: makepkg, repo-add and package signing all run inside containers, so it needs only Docker, rclone, bsdtar, jq, git and rsync. Docker is left alone when it already works, rather than replacing a working installation from Docker's own repository with the distribution's.

bin/repo setup --check         # Report what is missing, change nothing
bin/repo setup --skip-timers   # Prepare the host without the release timers

Signing credentials (/root/.omarchy/build-credentials) and the rclone remote hold secrets, so setup reports on them rather than creating them.

Management

Abandoned. Kept as reference for the tooling; don't follow it to publish.

# Everything at a glance: schedules, last runs, queues, backoff, lock
bin/repo timers                     # forwards to the host when one is configured

# Manual trigger (edge, rc, stable)
systemctl start omarchy-check-versions.service
systemctl start omarchy-auto-release-edge.service

# View logs
journalctl -u omarchy-auto-release-edge.service -n 50

# Clear a channel stuck in failure backoff (a new commit also clears it)
rm /root/.state/.build-failed-edge-x86_64

# Release lock left behind by a killed run (bin/repo timers shows if it is live)
rm /root/omarchy-pkgs/pkgs.omarchy.org/.release.lock

With the timers enabled, check-versions pulls the main checkout on its 5-minute tick. With them disabled, nothing does: bin/repo pulls before a command it forwards, and a command run on the host itself needs a git pull first. The checkout must be on master.