Setting the file. One moment.
Subchapter 27.42
audio/references/remove-background.mdMarkdown8 KBView on GitHub
Make a transparent overlay (typical: a talking head over an arbitrary scene). Uses u2net_human_seg (MIT).
scripts/11 filesnpx hyperframes remove-background subject.mp4 -o transparent.webm # default: VP9 + alpha
npx hyperframes remove-background subject.mp4 -o transparent.mov # ProRes 4444 (editing)
npx hyperframes remove-background portrait.jpg -o cutout.png # single-image cutout
npx hyperframes remove-background subject.mp4 -o subject.webm \
--background-output plate.webm # both layers, one pass
npx hyperframes remove-background subject.mp4 -o transparent.webm --device cpu
npx hyperframes remove-background --info # detected providers.webm (VP9 alpha) — default. Plug straight into <video> for Chrome-native transparent playback (~1 MB / 4s @ 1080p)..mov (ProRes 4444) — round-trip in editors (Premiere / Resolve / DaVinci). ~50 MB / 4s..png — single-image cutout.Controls VP9 encoder CRF only — segmentation quality is fixed. Higher quality keeps the cutout’s RGB closer to the source MP4 (important when overlaying the cutout on its own source).
| Preset | CRF | When |
|---|---|---|
fast | 30 | Iterating, smaller files, looser color match |
balanced | 18 | Default; visually identical for most uses |
best | 12 | Master / final delivery, tightest color match |
auto (default) picks CoreML on Apple Silicon, CUDA when available, otherwise CPU. Force with --device cpu | coreml | cuda. CUDA requires HYPERFRAMES_CUDA=1 plus a GPU-enabled onnxruntime-node build. Use --info to inspect detected providers without rendering.
The cutout WebM is a re-encoded copy of the source MP4’s RGB. What sits behind it matters.
| Pattern | Behind the cutout | Result |
|---|---|---|
| Cutout over a different scene (most common) | Static image, gradient, unrelated video | Looks great. Single RGB source for the subject. |
| Cutout over its own source mp4 (text-behind-subject) | Same mp4 the cutout came from | At balanced doubling is barely visible; at fast you’ll see color shift / edge halo. Use best for masters. |
| Cutout over a different take of the same person | Footage of the same subject | Two overlapping people. Don’t do this. |
Putting a headline behind a presenter cutout:
<video
src="presenter.mp4"
id="bg"
data-start="0"
data-duration="6"
data-track-index="0"
muted
playsinline
></video>
<h1 id="headline" style="z-index:2; ...">MAKE IT IN HYPERFRAMES</h1>
<div class="cutout-wrap" style="position:absolute; inset:0; z-index:3; opacity:0">
<video
src="presenter.webm"
data-start="0"
data-duration="6"
data-track-index="1"
muted
playsinline
></video>
</div>// Flip the wrapper's opacity at the cut, NOT the video's
tl.set(".cutout-wrap", { opacity: 1 }, 3.3);Two rules that are easy to miss:
<video> in a non-timed <div> and animate the wrapper’s opacity, not the video element’s. The framework forces opacity: 1 on active clips (any element with data-start / data-duration), so animating the video’s opacity directly is silently overridden. The wrapper has no data-* attributes, so it’s owned by your CSS / GSAP.data-start="0" and data-media-start="0" so the framework decodes them in sync from t=0. Late-mounting the cutout (data-start=3.3) introduces a seek + warm-up that lands a frame off the base mp4 — visible as one frame of misalignment at the cut.Emits a second transparent video alongside the cutout: same source RGB, alpha is 255 - mask instead of mask. The cutout has the subject opaque; the plate has the surroundings opaque (with a transparent hole where the subject was). Use it when text / graphics need to live between the two layers.
| File | Alpha is… | Use it for |
|---|---|---|
-o subject.webm | mask — subject opaque, background transparent | Foreground layer (top) |
--background-output plate.webm | inverse mask — surroundings opaque, subject transparent | Bottom layer; place text / graphics between this and the subject |
Both share the same --quality and run from a single inference pass — only encode cost roughly doubles. Only valid for video inputs with .webm / .mov outputs.
Hole-cut, not inpainted. The subject region in plate.webm is fully transparent — composite something opaque under it to fill the hole.
Single test for whether --background-output is the right tool: will anything ever be visible through the subject’s silhouette where the subject used to be? If no, you don’t need the plate — subject.webm alone over a different background is enough.
| Use case | Right tool |
|---|---|
| Text/graphics between the cutout and the plate (this command’s reason for existing) | Hole-cut (--background-output) |
| Subject onto an unrelated scene | Just subject.webm; ignore the plate |
| Show the room without the person, alone over no other content | Clean plate — needs an inpainter (LaMa, ProPainter, E2FGVI). Not this command. |
| Replace the subject with a different subject | Clean plate — same as above |
Ship just the two transparent layers and let arbitrary content live between them — no original mp4 needed:
<!-- z=1 plate: surroundings opaque, subject silhouette transparent -->
<video
src="plate.webm"
data-start="0"
data-duration="6"
data-track-index="0"
muted
playsinline
></video>
<!-- z=2 your content lives between the layers -->
<h1 id="headline" style="z-index:2; ...">MAKE IT IN HYPERFRAMES</h1>
<!-- z=3 cutout floats the subject back on top -->
<div class="cutout-wrap" style="position:absolute; inset:0; z-index:3">
<video
src="subject.webm"
data-start="0"
data-duration="6"
data-track-index="1"
muted
playsinline
></video>
</div>Functionally equivalent to the text-behind-subject pattern above, but doesn’t require shipping the original mp4 — the plate replaces it. Use this when delivering just the two transparent layers as a reusable asset.
If a user asks for “the room without the person, displayed standalone” (no subject anywhere, no compositing on top), --background-output is wrong — its plate has a transparent hole, not a filled-in clean plate. They need an inpainter: LaMa, ProPainter, or E2FGVI. Tell them this command can’t do it.