ZacharyZhang-NY
e74dafcd36
Three independently ugly bugs that all fed the same log flood:
* Electron's `WebFrameMain.send` writes every "Render frame was disposed"
to stderr via `console.error`. When the worker fans a second
agent-event past a reloading renderer, we produced thousands of
identical lines and stalled the main loop on stderr I/O. Added a
`mainFrame.detached` guard so the send is short-circuited before
Electron's own catch block fires, and installed a targeted filter in
`suppress-warnings.ts` (now actually imported from the entry point)
so the specific `console.error("Error sending from webFrameMain:"…)`
noise never reaches the terminal. Any other payload still surfaces.
* `dispatchAgentMessage` accepted the human-reader sentinel as a wake
target when an agent replied to `__board__` — `wakeAgentIfPossible`
then called `db.getAgent("__board__")`, threw, and the dispatcher
kept repeating on every send. Strip `HUMAN_SENDER_ID` at
`resolveMessageWakeTargets`, and keep a defense-in-depth guard in the
heartbeat so a stray caller (future automation fan-out) cannot
reintroduce the crash.
* Pi's skills loader rejects any SKILL.md whose frontmatter `name`
diverges from its parent directory. The four bundled `my-writing-style`
copies declared `name: writing-style-skill`, so every install emitted
a warning and dropped the skill. They also lived as orphan gitlinks
(mode 160000 without .gitmodules), and the global `*.md` gitignore
hid every other SKILL.md from version control. Fixed the frontmatter
to match the dir, converted the dangling submodules to plain trees,
and allowlisted `builtin-skills/**/*.md` so a fresh clone carries the
catalogue instead of silently losing it.
1.9 KiB
1.9 KiB
name, description, metadata
| name | description | metadata | ||
|---|---|---|---|---|
| transcribe-captions | Transcribing audio to generate captions in Remotion |
|
Transcribing audio
To transcribe audio to generate captions in Remotion, you can use the transcribe() function from the @remotion/install-whisper-cpp package.
Prerequisites
First, the @remotion/install-whisper-cpp package needs to be installed. If it is not installed, use the following command:
npx remotion add @remotion/install-whisper-cpp
Transcribing
Make a Node.js script to download Whisper.cpp and a model, and transcribe the audio.
import path from "path";
import {
downloadWhisperModel,
installWhisperCpp,
transcribe,
toCaptions,
} from "@remotion/install-whisper-cpp";
import fs from "fs";
const to = path.join(process.cwd(), "whisper.cpp");
await installWhisperCpp({
to,
version: "1.5.5",
});
await downloadWhisperModel({
model: "medium.en",
folder: to,
});
// Convert the audio to a 16KHz wav file first if needed:
// import {execSync} from 'child_process';
// execSync('ffmpeg -i /path/to/audio.mp4 -ar 16000 /path/to/audio.wav -y');
const whisperCppOutput = await transcribe({
model: "medium.en",
whisperPath: to,
whisperCppVersion: "1.5.5",
inputPath: "/path/to/audio123.wav",
tokenLevelTimestamps: true,
});
// Optional: Apply our recommended postprocessing
const { captions } = toCaptions({
whisperCppOutput,
});
// Write it to the public/ folder so it can be fetched from Remotion
fs.writeFileSync("captions123.json", JSON.stringify(captions, null, 2));
Transcribe each clip individually and create multiple JSON files.
See Displaying captions for how to display the captions in Remotion.