ea6ee9440a
The mute was reachable only as `omarchy-toggle crash-ignore/<program>`, which asks whoever runs it to know the flag layout, to reduce a binary's path to the name the watcher keys on, and to have read the rule that a name climbing out of that directory writes an unrelated toggle. All of that was carried in the skill's prose, which is the wrong place for a rule that has to hold: prose is advice, and the thing being advised about is a name the crashed program chose. So it is a command now. `omarchy crash mute hyprland` silences that program, `off` lifts it, `toggle` flips it, and no argument lists what is muted. It takes the binary's path as readily as the name and reduces it the way the watcher does, so the `Executable:` line from `coredumpctl` can be handed straight to it; it refuses what is not one component of a name, so it cannot be talked into writing outside its own directory whatever it is given; and it re-reads the flag afterwards and reports what is now true rather than what was asked for. The listing counts only regular files, because that is all the watcher honours -- anything else in there would read as muted while the crashes kept arriving. A leading `--` is consumed so a program named `-h`, which the router would otherwise answer with its own help, can still be muted. The watcher gained an unrelated fix that this uncovered. Its fields are read with `IFS=$'\t'`, and tab is IFS whitespace, so an empty field collapsed into the next delimiter and shifted every field after it along one: a crash whose comm was empty had a path read as its pid and was discarded as somebody else's. A process can set its comm to nothing, so that was reachable. Empty fields now arrive as a dash like missing ones, and a dash joins the empty and dot cases that fall back to `unknown`. Co-Authored-By: Codex XHigh <noreply@openai.com>
6.8 KiB
name, description
| name | description |
|---|---|
| diagnose-crash | Diagnose why a program crashed on this machine, from a systemd-coredump core dump. Use when a process has segfaulted, aborted, or otherwise dumped core, when asked why an application crashed or disappeared, or when a "Process crashed:" desktop notification is acted on. Triggers: crash, segfault, SIGSEGV, SIGABRT, core dump, coredumpctl, "why did X crash", "X keeps crashing", backtrace symbolization. Covers reporting a confirmed Omarchy bug upstream — see reporting.md. |
Diagnosing a Crash
Work from evidence. The goal is an honest account of what happened, not a plausible-sounding story.
Establish the facts
coredumpctl info <pid> is the starting point. Beyond the backtrace, note the
command line the process was started with — it usually reveals what the
program was working on when it died, which is often the whole answer.
coredumpctl list shows whether this crash is a one-off or a pattern. Repeated
crashes of the same program, or several programs dying together, point somewhere
different than a single failure does.
Rule out the boring causes first
Check resource exhaustion before blaming the program: free -h, and the journal
for OOM kills. A process killed by the OOM killer is not a bug in that process.
Correlate against the timeline
The crash timestamp is the most underused piece of evidence. Compare it against:
- Filesystem mtimes. A directory or file whose mtime lands on the same second as the crash strongly suggests what triggered it.
- The journal around that moment, for related warnings from the same or neighbouring processes.
- Recent package updates. A crash that starts right after an update points at the update.
Read the whole core, not just frame 0
Thread stacks other than the crashing one show what work was in flight — thumbnailers, image loaders, IPC readers, GPU queues. That context often explains the trigger even when the crashing frame itself cannot be symbolized.
Note any third-party code in the address space: file-manager or browser extensions, plugins, out-of-tree drivers. In-process third-party code is a common crash source and worth flagging — but do not pin blame on it without evidence that it is actually implicated.
Symbolize when you can
This is Arch, which runs a public debuginfod server:
core=$(mktemp -t crash-XXXXXX.core)
trap 'rm -f "$core"' EXIT
coredumpctl dump <pid> --output="$core"
DEBUGINFOD_URLS="https://debuginfod.archlinux.org" \
gdb -q <executable> "$core" \
-batch -ex 'set debuginfod enabled on' -ex 'bt'
A core is a verbatim copy of the process's memory and can hold passwords, tokens,
and private documents. Write it to a fresh mktemp path rather than a predictable
shared one, and delete it when you are done — never leave it lying in /tmp.
Many packages publish no debug symbols. When frames stay unresolved, say so — never invent function names to fill the gap. An unsymbolized stack still has shape: which library each frame belongs to, and whether the crash came from a signal handler, a main loop, or a worker thread.
Report
- What crashed, and what it was doing at the time.
- The most likely mechanism — separating clearly what the evidence proves from what you are inferring.
- Whether any user data was lost, and where it can be recovered from. Check the trash before concluding anything is gone.
- Whether it is likely to recur, and what would avoid or fix it.
Be straight about the limits of the evidence. If the cause is genuinely ambiguous, say so rather than assembling confidence out of guesswork.
Leave the system as you found it. Diagnosis reads; it does not fix, tidy, or reconfigure. The one thing to clean up is your own: delete the core you extracted above, which is a copy of the crashed process's memory. The single change a diagnosis may make is the mute below, and only when the user asks for it.
Offer to stop the notifications for this program
A crash that is now understood keeps announcing itself, and understanding it rarely stops it happening: an upstream bug waiting on a release, a program that dumps core every time it exits, a driver that misbehaves on this hardware. Finish by offering to silence crash notifications for that one program:
omarchy-crash-mute '<program>'
<program> is the process: name in the crash facts, or the binary: path —
the command reduces a path to the same name the watcher keys on, so passing
/usr/lib/chromium/chromium-browser and passing chromium-browser land on the
same flag. Prefer the binary's path wherever the crash recorded one: the kernel
truncates the process name to 15 characters and does not truncate a basename, and
a mute on the truncated form matches nothing, forever, while looking like it
worked.
That is also the answer for a diagnosis started by hand from omarchy agent crash <pid>, which is handed no name at all: give the command the Executable: line
from coredumpctl info and let it do the reducing. Some crashes record no
executable — pass the process name then, and unknown where the crash has
neither, which is the name such a crash is announced under.
The name is still whatever the crashed program's author chose to call a file, so
handle it as hostile text rather than as a word. The command refuses a name that
is not one — it cannot be talked into writing a flag outside its own directory —
but that is no help if the name reaches a shell unescaped first: single quotes
hold a space or a $(...), and a name containing a single quote closes them and
runs the rest as your shell. Escape it, or the program that just crashed picks
the command.
Offer it; never run it unprompted. The user may well want to keep being told.
Say how to undo it in the same breath, so it is not a one-way door — and the command answers both halves itself:
omarchy-crash-mute '<program>' off # un-mute this one
omarchy-crash-mute # list what is muted
The key is a bare name, so programs sharing one share a mute, and anything run
through an interpreter is keyed as the interpreter. Muting python3.13 or node
silences every other Python or Node program on the machine, which is rarely what
the user means: say so rather than quietly doing it.
This silences one program. Every other crash still notifies, and the muted program still crashes — nothing here fixes anything, and a mute offered instead of a fix that was within reach is the wrong answer. If the user wants crash notifications off altogether, that is Trigger > Toggle > Crash Capture instead.
If it is an Omarchy bug
Most application crashes are upstream bugs in those applications, not Omarchy's
doing. In the minority of cases where the cause really does sit within Omarchy's
sphere of control, read reporting.md before offering to file
anything.