0 Fork 0

Files

T

ZacharyZhang-NY e74dafcd36 Silence render-disposed spam, unbreak bundled skills, drop board wake

Three independently ugly bugs that all fed the same log flood:

* Electron's `WebFrameMain.send` writes every "Render frame was disposed"
  to stderr via `console.error`. When the worker fans a second
  agent-event past a reloading renderer, we produced thousands of
  identical lines and stalled the main loop on stderr I/O. Added a
  `mainFrame.detached` guard so the send is short-circuited before
  Electron's own catch block fires, and installed a targeted filter in
  `suppress-warnings.ts` (now actually imported from the entry point)
  so the specific `console.error("Error sending from webFrameMain:"…)`
  noise never reaches the terminal. Any other payload still surfaces.

* `dispatchAgentMessage` accepted the human-reader sentinel as a wake
  target when an agent replied to `__board__` — `wakeAgentIfPossible`
  then called `db.getAgent("__board__")`, threw, and the dispatcher
  kept repeating on every send. Strip `HUMAN_SENDER_ID` at
  `resolveMessageWakeTargets`, and keep a defense-in-depth guard in the
  heartbeat so a stray caller (future automation fan-out) cannot
  reintroduce the crash.

* Pi's skills loader rejects any SKILL.md whose frontmatter `name`
  diverges from its parent directory. The four bundled `my-writing-style`
  copies declared `name: writing-style-skill`, so every install emitted
  a warning and dropped the skill. They also lived as orphan gitlinks
  (mode 160000 without .gitmodules), and the global `*.md` gitignore
  hid every other SKILL.md from version control. Fixed the frontmatter
  to match the dir, converted the dangling submodules to plain trees,
  and allowlisted `builtin-skills/**/*.md` so a fresh clone carries the
  catalogue instead of silently losing it.

2026-04-20 08:13:48 -04:00

3.2 KiB

Raw Blame History

name, description, metadata

name

description

metadata

voiceover

Adding AI-generated voiceover to Remotion compositions using TTS

tags
voiceover, audio, elevenlabs, tts, speech, calculateMetadata, dynamic duration

Adding AI voiceover to a Remotion composition

Use ElevenLabs TTS to generate speech audio per scene, then use calculateMetadata to dynamically size the composition to match the audio.

Prerequisites

By default this guide uses ElevenLabs as the TTS provider (ELEVENLABS_API_KEY environment variable). Users may substitute any TTS service that can produce an audio file.

If the user has not specified a TTS provider, recommend ElevenLabs and ask for their API key.

Ensure the environment variable is available when running the generation script:

node --strip-types generate-voiceover.ts

Generating audio with ElevenLabs

Create a script that reads the config, calls the ElevenLabs API for each scene, and writes MP3 files to the public/ directory so Remotion can access them via staticFile().

The core API call for a single scene:

const response = await fetch(
  `https://api.elevenlabs.io/v1/text-to-speech/${voiceId}`,
  {
    method: "POST",
    headers: {
      "xi-api-key": process.env.ELEVENLABS_API_KEY!,
      "Content-Type": "application/json",
      Accept: "audio/mpeg",
    },
    body: JSON.stringify({
      text: "Welcome to the show.",
      model_id: "eleven_multilingual_v2",
      voice_settings: {
        stability: 0.5,
        similarity_boost: 0.75,
        style: 0.3,
      },
    }),
  },
);

const audioBuffer = Buffer.from(await response.arrayBuffer());
writeFileSync(`public/voiceover/${compositionId}/${scene.id}.mp3`, audioBuffer);

Dynamic composition duration with calculateMetadata

Use calculateMetadata to measure the audio durations and set the composition length accordingly.

import { CalculateMetadataFunction, staticFile } from "remotion";
import { getAudioDuration } from "./get-audio-duration";

const FPS = 30;

const SCENE_AUDIO_FILES = [
  "voiceover/my-comp/scene-01-intro.mp3",
  "voiceover/my-comp/scene-02-main.mp3",
  "voiceover/my-comp/scene-03-outro.mp3",
];

export const calculateMetadata: CalculateMetadataFunction<Props> = async ({
  props,
}) => {
  const durations = await Promise.all(
    SCENE_AUDIO_FILES.map((file) => getAudioDuration(staticFile(file))),
  );

  const sceneDurations = durations.map((durationInSeconds) => {
    return durationInSeconds * FPS;
  });

  return {
    durationInFrames: Math.ceil(sceneDurations.reduce((sum, d) => sum + d, 0)),
  };
};

The computed sceneDurations are passed into the component via a voiceover prop so the component knows how long each scene should be.

If the composition uses <TransitionSeries>, subtract the overlap from total duration: ./transitions.md#calculating-total-composition-duration

Rendering audio in the component

See audio.md for more information on how to render audio in the component.

Delaying audio start

See audio.md#delaying for more information on how to delay the audio start.

3.2 KiB Raw Blame History