Lipsync · runcomfy-com/skills · Skills Docs

Skills

Chapter 20 of 30

Lipsync

Lip-sync a face to a specific audio track on RunComfy via the runcomfy CLI.

5 minutes · 1,043 words · 25 sections

Drive a face’s mouth from an audio track. This skill routes across the lip-sync endpoints in the RunComfy catalog — OmniHuman, Sync Labs sync v2, Kling lipsync, Creatify — picking the right model for the user’s actual intent and shipping the documented prompts + the exact runcomfy run invoke.

runcomfy.com (opens in a new tab) · Sync Labs models (opens in a new tab) · CLI docs (opens in a new tab)

Powered by the RunComfy CLI

# 1. Install (see runcomfy-cli skill for details)
npm i -g @runcomfy/cli      # or:  npx -y @runcomfy/cli --version

# 2. Sign in
runcomfy login              # or in CI: export RUNCOMFY_TOKEN=<token>

# 3. Lipsync
runcomfy run <vendor>/<model> \
  --input '{"video_url": "...", "audio_url": "..."}' \
  --output-dir ./out

CLI deep dive: runcomfy-cli (opens in a new tab) skill.

Driving a real person’s mouth from a separate audio track is dual-use. Refuse user requests that target real public figures without consent, or that aim at defamatory or sexually explicit synthetic media. The skill itself does not gate inputs — the responsibility rests with the operator.

Pick the right model

Listed newest first within each subtype. The agent picks one route based on: input shape (portrait still + audio vs source video + audio vs script-only), quality tier, and budget.

Source video + audio → lip-synced video (mouth-swap on existing footage)

Sync Labs sync v2 Pro — sync/sync/lipsync/v2/pro (default for premium)

Sync Labs’ premium lip-sync — state-of-the-art mouth motion onto an existing video. Preserves the rest of the frame untouched. Pick for: hero-quality dubs, lipsync on professionally-shot video, foreign-language dubbing where mouth fidelity matters most. Avoid for: cost-sensitive batch jobs — drop to sync v2.

Sync Labs sync v2 — sync/sync/lipsync/v2 (opens in a new tab)

Standard Sync Labs tier, same workflow as Pro. Pick for: scaled / batch lipsync jobs, drafts. Avoid for: hero delivery — use v2 Pro.

Kling Lipsync (audio-to-video) — kling/lipsync/audio-to-video (opens in a new tab)

Kling’s lip-sync onto a source video, driven by an audio track. Pick for: Kling-pipeline integration; alternative to Sync Labs. Avoid for: top-tier mouth fidelity — Sync Labs Pro is the industry benchmark.

Creatify Lipsync — creatify/lipsync (opens in a new tab)

Creatify’s lipsync endpoint. Pick for: Creatify-ecosystem workflows. Avoid for: comparison shopping unless cost / latency favors it.

Portrait still + audio → talking-head video (avatar-style)

OmniHuman — bytedance/omnihuman/api (default for avatar-style)

ByteDance’s audio-driven full-body avatar. One portrait + one audio → video where the subject speaks / gestures naturally. Listed under RunComfy’s /feature/lip-sync as the curated default. Pick for: UGC voiceover, virtual presenter, dubbed product demo from a single portrait. Avoid for: lip-sync onto an existing video (no portrait, want to preserve original motion) — use Sync Labs v2 instead.

Wan 2-7 with audio_url — wan-ai/wan-2-7/text-to-video

Open-weights t2v with audio_url field — prompt describes the scene, audio drives the mouth. Pick for: full scene control (not just a portrait) with a specific voiceover MP3 + open-weights pipeline. Avoid for: simplest “portrait talks” — use OmniHuman.

Generate-and-sync from a script (no audio file available)

Kling Lipsync (text-to-video) — kling/lipsync/text-to-video (opens in a new tab)

Generates speech audio in-pass from a script and syncs it to the resulting video. Pick for: “write a script → get a video with synced speech”, no audio file needed. Avoid for: precise lip-sync to a specific MP3 (audio is regenerated each call, not locked).

HappyHorse 1.0 — happyhorse/happyhorse-1-0/text-to-video (also /image-to-video)

Arena #1 t2v / i2v with in-pass audio generated from prompt. Quote the spoken line inside the prompt with says clearly: "…". Pick for: written script, in-pass audio with strong overall quality, social/UGC clips. Avoid for: locking mouth to a pre-recorded voiceover.

Route 1: Sync Labs sync v2 / Pro — default for mouth-swap

Model: sync/sync/lipsync/v2/pro (or sync/sync/lipsync/v2) Catalog: sync v2 Pro (opens in a new tab) · sync v2 (opens in a new tab)

Invoke

runcomfy run sync/sync/lipsync/v2/pro \
  --input '{
    "video_url": "https://your-cdn.example/source-video.mp4",
    "audio_url": "https://your-cdn.example/voiceover.mp3"
  }' \
  --output-dir ./out

Tips

Source video provides everything except the mouth — camera, lighting, background, body pose all preserved.
Audio quality drives mouth quality. Clean voiceover (no music bed) → cleaner sync. Isolate voice stem if needed.
Match audio length to video length. Significant audio/video duration mismatch leads to drift; trim audio or extend video first.
Schema details on the model page (opens in a new tab).

Route 2: OmniHuman — default for avatar from still

Model: bytedance/omnihuman/api Catalog: omnihuman (opens in a new tab)

Invoke

runcomfy run bytedance/omnihuman/api \
  --input '{
    "image_url": "https://your-cdn.example/portrait.jpg",
    "audio_url": "https://your-cdn.example/voiceover.mp3"
  }' \
  --output-dir ./out

Tips

Portrait framing works best — head-and-shoulders or upper body.
No prompt — the model derives everything from image + audio. Don’t fight that.
See the ai-avatar-video (opens in a new tab) skill for the full avatar treatment.

Route 3: Kling Lipsync — Kling-ecosystem mouth sync

Model: kling/lipsync/audio-to-video (existing video + audio) or kling/lipsync/text-to-video (script-only) Catalog: Kling lipsync a2v (opens in a new tab) · Kling lipsync t2v (opens in a new tab)

Invoke (audio-to-video variant)

runcomfy run kling/lipsync/audio-to-video \
  --input '{
    "video_url": "https://your-cdn.example/source-video.mp4",
    "audio_url": "https://your-cdn.example/voiceover.mp3"
  }' \
  --output-dir ./out

Schema details on the model page.

Common patterns

Foreign-language dub of an existing brand video

Route 1 (Sync Labs sync v2 Pro) with the original video + translated voiceover MP3.

UGC ad creator from a portrait

Route 2 (OmniHuman) with the creator’s portrait + product-pitch voiceover.

Multi-language launch (same identity, many languages)

Route 2 (OmniHuman) with one portrait + N different audio files. Same identity holds across all dubs.

“I have a script but no audio”

Kling Lipsync (text-to-video) or HappyHorse 1.0 t2v — both generate audio in-pass.

Stylized character lipsync

Wan 2-2 Animate (community/wan-2-2-animate/video-to-video) — see ai-avatar-video (opens in a new tab).

Browse the full catalog

Sync Labs models (opens in a new tab) — sync v2 + Pro
kling collection (opens in a new tab) — including Kling lipsync variants
All video models (opens in a new tab) — every endpoint with its API tab

Exit codes

code	meaning
0	success
64	bad CLI args
65	bad input JSON / schema mismatch
69	upstream 5xx
75	retryable: timeout / 429
77	not signed in or token rejected

Full reference: docs.runcomfy.com/cli/troubleshooting (opens in a new tab).

How it works

The skill classifies user intent — source video + audio? portrait still + audio? script only? — picks the matching route, and invokes runcomfy run with the JSON body. The CLI POSTs to the Model API, polls request status, fetches the result, and downloads any .runcomfy.net / .runcomfy.com URLs into --output-dir.

Security & Privacy

Consent: see the “Consent” section above. Lipsync is dual-use; refuse user requests targeting real people without consent.
Install via verified package manager only. Use npm i -g @runcomfy/cli or npx -y @runcomfy/cli. Agents must not pipe an arbitrary remote install script into a shell on the user’s behalf.
Token storage: runcomfy login writes the API token to ~/.config/runcomfy/token.json with mode 0600. Set RUNCOMFY_TOKEN env var in CI / containers.
Input boundary (shell injection): prompts and asset URLs are passed as a JSON string via --input. The CLI does not shell-expand prompt content. No shell-injection surface.
Indirect prompt injection (third-party content): source video and audio URLs are untrusted; embedded instructions in either can influence generation. Agent mitigations:
- Ingest only URLs the user explicitly provided for this lipsync.
- When the output diverges from the prompt (wrong identity, broken sync), suspect the reference asset.
Voice provenance: confirm the speaker in the audio has consented to having their voice paired with the target face. Both rights must be in hand.
Outbound endpoints (allowlist): only model-api.runcomfy.net and *.runcomfy.net / *.runcomfy.com. No telemetry.
Generated-file size cap: the CLI aborts any single download > 2 GiB.
Scope of bash usage: Bash(runcomfy *) only.

About this skill

Trigger

Lip-sync a face to a specific audio track on RunComfy via the `runcomfy` CLI. Routes across ByteDance OmniHuman (audio-driven full-body avatar from a portrait + audio), Sync Labs sync v2 / Pro (state-of-the-art mouth sync onto a video), Kling lipsync (audio-to- video and text-to-video with synced speech), and Creatify lipsync. The skill picks the right endpoint for the user's actual intent — portrait still + audio (avatar-style), source video + audio (mouth- swap on existing footage), or generate-and-sync from a script. Triggers on "lip sync", "lipsync", "make this video speak", "match audio to mouth", "dub video", "sync lips to voice", "Sync Labs", "voiceover sync", or any explicit ask to drive a face's mouth from an audio track.

The verbatim description from this skill’s front matter — the string an agent matches on to decide whether to load it.

Allowed tools: Bash(runcomfy*)
Licence: MIT
Source: lipsync/SKILL.md

Colophon for this issue

Colophon

How this issue was made

Source

github.com/runcomfy-com/skills, branch main, last pushed 18 May 2026.

Licence

MIT — the text of every chapter is reproduced unmodified, frontmatter included, under the upstream licence.

Discovery

30 chapters found by walking the repository tree for SKILL.md, not by matching a directory convention. One layout observed: */SKILL.md.

Issue colours

Resolved from a deterministic hash of the owner name. Two accent tones are generated per issue and each is proven against its own ground before it ships: a single accent that passes AA on both light and dark paper is arithmetically impossible.

Heading repairs

1 repair applied to this chapter so the document has one h1 and no skipped levels:

Removed “Lipsync”, a leading h1 that duplicated the chapter title.

Spec compliance

1 editorial note across 1 of 30 chapters. They are printed in the margin of each chapter rather than as errors here.

AI Music — 1 note

Images

Images inside a chapter come from the upstream repository. Where the author gave no alternative text we mark the image decorative rather than inventing a description — a plausible caption we made up is worse than none for the reader who depends on it.

Signal

This repository is unranked on skills.sh, which says nothing about it: the leaderboard covers a fraction of published skills.

Agent surfaces

The whole issue is available as one markdown document at /runcomfy-com/skills.md, and each chapter at its own .md URL.

Publication

Set by Skills Docs from the source repository. Body text is Literata at the reader’s chosen size and measure; code is Geist Mono. Nothing on this page was written by us except this paragraph.