--- title: "elevenlabs/skills" description: "Collections of skills for building with ElevenLabs" source: https://github.com/elevenlabs/skills ref: main license: MIT licenseName: "MIT License" canonical: https://skillsdocs.com/elevenlabs/skills base: https://github.com/elevenlabs/skills/blob/main/ provenance: mixed chapters: 11 inlined: 11 withheld: 0 words: 7756 updated: 2026-09-16T18:01:21Z generator: "Skills Docs" --- > **elevenlabs/skills** — every Agent Skill in this repository, inlined verbatim. > > Canonical HTML: https://skillsdocs.com/elevenlabs/skills > Per-skill Markdown: https://skillsdocs.com/elevenlabs/skills/.md > Machine manifest: https://skillsdocs.com/elevenlabs/skills/.well-known/agent-skills/index.json > JSON: https://skillsdocs.com/api/v1/books/elevenlabs/skills > Install: `npx skills add elevenlabs/skills` > Upstream: https://github.com/elevenlabs/skills @ `main` > Licence: MIT > > Content is mirrored from GitHub and © its authors, served unmodified. Takedown: https://github.com/DreambaseAI/skillsdocs/issues/new?labels=takedown&title=Takedown+request # elevenlabs/skills Collections of skills for building with ElevenLabs - **Skills:** 11 - **Authorship:** mixed — 1 of 11 are credited — skills in use here, not published from here - **Inlined:** 11 (licence detected) - **Words:** 7,756 - **Reading time:** 35 min - **Stars:** 459 ## Table of contents 1. [agents](https://skillsdocs.com/elevenlabs/skills/agents.md) — Build voice AI agents with ElevenLabs. Use when creating voice assistants, customer service bots, interactive voice characters, or any real-time voice conversa… 2. [dubbing](https://skillsdocs.com/elevenlabs/skills/dubbing.md) — Dub audio and video into other languages using the ElevenLabs Dubbing API (dubbing_v2), preserving the original speakers' voices. Use when translating videos,… 3. [music](https://skillsdocs.com/elevenlabs/skills/music.md) — Generate music using ElevenLabs Music API. Use when creating instrumental tracks, songs with lyrics, background music, jingles, or any AI-generated music compo… 4. [setup-api-key](https://skillsdocs.com/elevenlabs/skills/setup-api-key.md) — Guides users through setting up an ElevenLabs API key for REST API and SDK workflows. Use when the user needs to configure an ElevenLabs API key or when an API… 5. [sound-effects](https://skillsdocs.com/elevenlabs/skills/sound-effects.md) — Generate sound effects from text descriptions using ElevenLabs. Use when creating sound effects, generating audio textures, producing ambient sounds, cinematic… 6. [speech-engine](https://skillsdocs.com/elevenlabs/skills/speech-engine.md) — Add real-time voice conversations to a custom agent runtime with ElevenLabs Speech Engine. Use when building Speech Engine servers, WebSocket handlers, WebRTC… 7. [speech-to-text](https://skillsdocs.com/elevenlabs/skills/speech-to-text.md) — Transcribe audio to text using ElevenLabs Scribe v2. Use when converting audio/video to text, generating subtitles, transcribing meetings, or processing spoken… 8. [text-to-speech](https://skillsdocs.com/elevenlabs/skills/text-to-speech.md) — Convert text to speech using ElevenLabs voice AI. Use when generating audio from text, creating voiceovers, building voice apps, or synthesizing speech in 70+… 9. [voice-changer](https://skillsdocs.com/elevenlabs/skills/voice-changer.md) — Transform the voice in an audio recording into a different target voice while preserving emotion, timing, and delivery using the ElevenLabs Voice Changer (spee… 10. [voice-isolator](https://skillsdocs.com/elevenlabs/skills/voice-isolator.md) — Remove background noise and isolate vocals/speech from audio using ElevenLabs Voice Isolator (audio isolation) API. Use when cleaning up noisy recordings, remo… 11. [update-skills-from-changelog](https://skillsdocs.com/elevenlabs/skills/update-skills-from-changelog.md) — Update ElevenLabs agent skills from a merged weekly changelog in elevenlabs-dx, then open a pull request in elevenlabs/skills. Trigger after a changelog merges… ## Front matter _The repository README, verbatim except that relative links are resolved against https://github.com/elevenlabs/skills/blob/main/._ ![LOGO](https://github.com/elevenlabs/skills/blob/main/logo.png) # ElevenLabs Skills Agent skills for [ElevenLabs](https://elevenlabs.io) developer products. These skills follow the [Agent Skills specification](https://agentskills.io/specification) and can be used with any compatible AI coding assistant. ## Installation ```bash npx skills add elevenlabs/skills ``` ## Available Skills | Skill | Description | |-------|-------------| | [text-to-speech](https://github.com/elevenlabs/skills/blob/main/text-to-speech) | Convert text to lifelike speech using ElevenLabs' AI voices | | [speech-to-text](https://github.com/elevenlabs/skills/blob/main/speech-to-text) | Transcribe audio files to text with timestamps | | [speech-engine](https://github.com/elevenlabs/skills/blob/main/speech-engine) | Add real-time voice conversations to a custom LLM or chat agent | | [agents](https://github.com/elevenlabs/skills/blob/main/agents) | Build conversational voice AI agents | | [sound-effects](https://github.com/elevenlabs/skills/blob/main/sound-effects) | Generate sound effects from text descriptions | | [music](https://github.com/elevenlabs/skills/blob/main/music) | Generate music tracks using AI composition | | [voice-changer](https://github.com/elevenlabs/skills/blob/main/voice-changer) | Transform the voice in an audio recording into a different target voice (speech-to-speech) | | [voice-isolator](https://github.com/elevenlabs/skills/blob/main/voice-isolator) | Remove background noise and isolate vocals/speech from audio | | [dubbing](https://github.com/elevenlabs/skills/blob/main/dubbing) | Dub audio/video into other languages while preserving the original speakers' voices | | [setup-api-key](https://github.com/elevenlabs/skills/blob/main/setup-api-key) | Guide through obtaining and configuring an ElevenLabs API key | ## Configuration All skills require an ElevenLabs API key. Set it as an environment variable: ```bash export ELEVENLABS_API_KEY="your-api-key" ``` Get your API key from the `setup-api-key` skill or use the [ElevenLabs dashboard](https://elevenlabs.io/app/settings/api-keys). ## SDK Support Most skills include examples for: - **Python** - `pip install elevenlabs` - **JavaScript/TypeScript** - `npm install @elevenlabs/elevenlabs-js` - **CLI** - `npm install -g @elevenlabs/cli` or `brew install elevenlabs/tap/elevenlabs` (wraps the REST API; reads `ELEVENLABS_API_KEY` automatically) > **JavaScript SDK Warning:** Always use `@elevenlabs/elevenlabs-js`. Do not use `npm install elevenlabs` (that's an outdated v1.x package). See the installation guide in any skill's `references/` folder for complete setup instructions including migration from deprecated packages. ## Evaluations The `evals/` directory contains trigger and functional evaluations for all skills. ```bash # Run all evaluations (trigger + functional) python3 evals/run_all.py -v # Trigger evals only — tests if skills fire for the right queries (~3 min) python3 evals/run_all.py --trigger-only -v # Functional evals only — tests if skills produce correct output (~15 min) python3 evals/run_all.py --functional-only -v # Specific skills python3 evals/run_all.py --skills text-to-speech agents -v # Custom model (see `cursor-agent --list-models`) python3 evals/run_all.py --model gpt-5.4-high -v ``` Results are saved to `evals/results//` with a `report.md` summary and `results.json` for programmatic access. Functional evals use an isolated `cursor-agent` workspace per test case (under that results tree); they do **not** modify skill sources under each skill’s directory. Requires the [Cursor Agent CLI](https://cursor.com/docs/cli/using) (`cursor-agent` on your `PATH`; override binary with `CURSOR_AGENT`) and Cursor authentication (`cursor-agent login` or `CURSOR_API_KEY`). ## License MIT --- ## Part: Skills --- ## 1. agents - **Source:** https://github.com/elevenlabs/skills/blob/main/agents/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/agents/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/agents.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html Bundled files (7), referenced from this skill's directory: - `references/agent-configuration.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/agents/references/agent-configuration.md - `references/client-tools.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/agents/references/client-tools.md - `references/installation.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/agents/references/installation.md - `references/outbound-calls.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/agents/references/outbound-calls.md - `references/using-procedure-api.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/agents/references/using-procedure-api.md - `references/widget-embedding.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/agents/references/widget-embedding.md - `references/writing-procedures.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/agents/references/writing-procedures.md --- name: agents description: Build voice AI agents with ElevenLabs. Use when creating voice assistants, customer service bots, interactive voice characters, or any real-time voice conversation experience, and when configuring an agent's tools, workflows, or procedures, including creating, editing, compiling, and publishing procedure drafts on an agent branch over the SDKs or REST API. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Agents Platform Build voice AI agents with natural conversations, multiple LLM providers, custom tools, and easy web embedding. > **Setup:** See [Installation Guide](references/installation.md) for CLI and SDK setup. ## Quick Start with CLI The ElevenLabs CLI is the recommended way to create and manage agents: ```bash # Install CLI and authenticate npm install -g @elevenlabs/cli elevenlabs auth login # Initialize project and create an agent elevenlabs agents init elevenlabs agents add "My Assistant" --template complete # Push to ElevenLabs platform elevenlabs agents push ``` **Available templates:** `complete`, `minimal`, `voice-only`, `text-only`, `customer-service`, `assistant` ### Python ```python from elevenlabs import ElevenLabs client = ElevenLabs() agent = client.conversational_ai.agents.create( name="My Assistant", conversation_config={ "agent": { "first_message": "Hello! How can I help?", "language": "en", "prompt": { "prompt": "You are a helpful assistant. Be concise and friendly.", "llm": "gemini-2.0-flash", "temperature": 0.7 } }, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"} } ) ``` ### JavaScript ```javascript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; const client = new ElevenLabsClient(); const agent = await client.conversationalAi.agents.create({ name: "My Assistant", conversationConfig: { agent: { firstMessage: "Hello! How can I help?", language: "en", prompt: { prompt: "You are a helpful assistant.", llm: "gemini-2.0-flash", temperature: 0.7 } }, tts: { voiceId: "JBFqnCBsd6RMkjVDRZzb" } } }); ``` ### CLI The CLI reads `ELEVENLABS_API_KEY` from the environment automatically: ```bash elevenlabs agents create \ --json '{"name": "My Assistant", "conversation_config": {"agent": {"first_message": "Hello!", "language": "en", "prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}' ``` ## Starting Conversations **Authenticated WebRTC:** Request a session token from your backend. The response includes both the token and the conversation ID: ```python session = client.conversational_ai.conversations.get_webrtc_token( agent_id="your-agent-id", ) print(session.token, session.conversation_id) ``` **Server-side (Python):** Get signed URL for client connection: ```python signed_url = client.conversational_ai.conversations.get_signed_url( agent_id="your-agent-id", environment="staging", ) ``` **Client-side (JavaScript):** ```javascript import { Conversation } from "@elevenlabs/client"; const conversation = await Conversation.startSession({ agentId: "your-agent-id", environment: "staging", overrides: { asr: { keywords: ["ElevenLabs", "TechCorp"] } }, onMessage: (msg) => console.log("Agent:", msg.message), onUserTranscript: (t) => console.log("User:", t.message), onPing: (event) => console.log("Estimated latency:", event.ping_ms), onContextUsage: ({ model, context_tokens, context_limit_tokens }) => console.log(`${model}: ${context_tokens}/${context_limit_tokens} context tokens`), onError: (e) => console.error(e) }); ``` **React Hook:** Wrap hook consumers in `ConversationProvider`. Prefer granular hooks such as `useConversationControls` and `useConversationStatus` for session controls and UI state; `useConversation` remains available as the convenience all-in-one hook. Pass provider-level callbacks such as `onError` when you want React to handle conversation errors in one place. ```typescript import { ConversationProvider, useConversationControls, useConversationStatus, } from "@elevenlabs/react"; function Agent({ signedUrl }: { signedUrl: string }) { const { startSession, endSession } = useConversationControls(); const { status } = useConversationStatus(); if (status === "connected") { return ; } return ( ); } function App({ signedUrl }: { signedUrl: string }) { return ( console.error("Conversation error:", error)} onPing={(event) => console.log("Estimated latency:", event.ping_ms)} onContextUsage={({ model, context_tokens, context_limit_tokens }) => console.log(`${model}: ${context_tokens}/${context_limit_tokens} context tokens`) } > ); } ``` ## Configuration | Provider | Models | |----------|--------| | OpenAI | `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `gpt-5.5-2026-04-23`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.4-2026-03-05`, `gpt-5.4-mini-2026-03-17`, `gpt-5.4-nano-2026-03-17`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo` | | Anthropic | `claude-opus-4-7`, `claude-sonnet-4-6`, `claude-sonnet-4-5`, `claude-sonnet-4`, `claude-haiku-4-5`, `claude-3-7-sonnet`, `claude-3-5-sonnet`, `claude-3-haiku` | | Google | `gemini-3.7-flash`, `gemini-3.6-flash`, `gemini-3.1-flash-lite-preview`, `gemini-3.1-pro-preview`, `gemini-3-pro-preview`, `gemini-3-flash-preview`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash`, `gemini-2.0-flash-lite` | | ElevenLabs | `glm-45-air-fp8`, `qwen3-30b-a3b`, `qwen36-35b-a3b`, `qwen35-35b-a3b`, `qwen35-397b-a17b`, `gpt-oss-120b` | | Custom | `custom-llm` (bring your own endpoint) | Use `GET /v1/convai/llm/list` to inspect the current model catalog, including deprecation state, token/context limits, capability flags such as image-input support, and model-specific reasoning effort support. **Popular voices:** `JBFqnCBsd6RMkjVDRZzb` (George), `EXAVITQu4vr4xnSDxMaL` (Sarah), `onwK4e9ZLuTAKqWW03F9` (Daniel), `XB0fDUnXU5powFXDhCwa` (Charlotte) **Turn eagerness:** `patient` (waits longer for user to finish), `normal`, or `eager` (responds quickly) See [Agent Configuration](references/agent-configuration.md) for all options. ## System Prompt Structure Section the prompt with markdown headings — the model prioritizes and interprets instructions more reliably ([prompting guide](https://elevenlabs.io/docs/eleven-agents/best-practices/prompting-guide)): ``` # Personality – named character, 2-3 traits # Environment – where they work, who they talk to # Tone – vocal style as 4-5 bullets # Goal – what success looks like (numbered for multi-step flows) ``` Keep instructions short and action-based. Mark critical steps with "This step is important." For critical refusal/safety rules, include concise instructions in the prompt and also configure independent custom Guardrails via `platform_settings.guardrails` (see [Guardrails](#guardrails)). ## Tools Extend agents with webhook, client, or built-in system tools. Tools are defined inside `conversation_config.agent.prompt`: Workspace environment variables can resolve per-environment server tool URLs, headers, and auth connections, and runtime system variables such as `{{system__conversation_history}}` can pass full conversation context into tool calls when needed. ```python "prompt": { "prompt": "You are a helpful assistant that can check the weather.", "llm": "gemini-2.0-flash", "tools": [ # Webhook: server-side API call {"type": "webhook", "name": "get_weather", "description": "Get weather", "api_schema": {"url": "https://api.example.com/weather", "method": "POST", "request_body_schema": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}, # Client: runs in the browser {"type": "client", "name": "show_product", "description": "Display a product", "parameters": {"type": "object", "properties": {"productId": {"type": "string"}}, "required": ["productId"]}} ], "built_in_tools": { "end_call": {}, "transfer_to_number": {"transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]}, "start_procedure": {} } } ``` **Client tools** run in browser: ```javascript clientTools: { show_product: async ({ productId }) => { document.getElementById("product").src = `/products/${productId}`; return { success: true }; } } ``` See [Client Tools Reference](references/client-tools.md) for complete documentation. ### Built-in System Tools Set under `conversation_config.agent.prompt.built_in_tools`. `{}` enables defaults; provide `description` to customize; omit to disable. | Tool | Enable for | |------|------------| | `end_call` | All agents | | `language_detection` | Multilingual agents | | `transfer_to_number` | Phone-based human escalation | | `transfer_to_agent` | Multi-agent workflows | | `start_procedure` | Procedure-guided conversations (see [Procedures](#procedures)) | | `end_procedure` | Completing active procedures | | `skip_turn` | Tutoring / coaching (silent listening) | | `voicemail_detection` | Outbound calling | | `play_keypad_touch_tone` | IVR navigation | `run_subagent` is a system tool for delegating a task to another configured agent. Add it to `conversation_config.agent.prompt.tools` with `params.system_tool_type: "run_subagent"` and an `agents` array. Each entry requires `agent_id` and `description`; `branch_id` and a JSON-schema `parameters` object are optional. `knowledge_base` is a system tool for letting the model choose how to inspect attached knowledge. Add it to `conversation_config.agent.prompt.tools` with `type: "system"`, a `name`, and `params.system_tool_type: "knowledge_base"`. Use `enabled_strategies` to expose any combination of `cat`, `keyword`, `semantic`, and `ls`: ```json { "type": "system", "name": "knowledge_base", "description": "Search the attached knowledge base.", "params": { "system_tool_type": "knowledge_base", "enabled_strategies": ["semantic", "keyword"] } } ``` ### Integration Tools Pre-built connectors managed by the platform. Create a connection with credentials, then attach via `tool_ids`: | Integration | Use case | |-------------|----------| | `calcom` | Scheduling appointments | | `salesforce` | CRM lookups, case creation | | `hubspot` | CRM, marketing, contacts | | `zendesk` | Support ticketing | Three-step flow: `POST /v1/convai/api-integrations/{id}/connections` → `GET /v1/convai/api-integrations/{id}/tools` → `POST /v1/convai/tools` with `api_integration_id` and `api_integration_connection_id`. Attach to the agent with `"prompt": {"tool_ids": ["tool_xxxx"]}`. Inline `tools` and `tool_ids` can coexist — prefer an integration over a duplicate custom webhook. ### Public-API Webhook Examples No-auth APIs useful for prototypes (URLs must be HTTPS): | Tool | URL | Purpose | |------|-----|---------| | `get_weather` | `https://wttr.in/{location}?format=j1` | Current weather | | `search_wikipedia` | `https://en.wikipedia.org/api/rest_v1/page/summary/{topic}` | Topic summary | | `get_exchange_rate` | `https://open.er-api.com/v6/latest/{base_currency}` | FX rates | ## Workflows Route conversations through discrete steps with branching logic. Define under the agent's top-level `workflow` field. Reference: [Agent Workflows](https://elevenlabs.io/docs/eleven-agents/customization/agent-workflows). **Node types:** `start` (ID must be `"start_node"`), `end`, `override_agent` (subagent step with `label` + `additional_prompt`), `dispatch_tool` (executes a tool with success/failure routing), `agent_transfer`, `transfer_to_number`. **Edge types:** `unconditional`, `llm` (natural-language condition), `expression` (deterministic data check). Tool nodes have separate success/failure edges. **Scope tools per step** with `additional_tool_ids` on a node — prevents the wrong tool firing at the wrong step. Set `additional_tool_ids: []` on conversational routing nodes such as greeting and `classify_intent` so they only converse: ```json { "type": "override_agent", "label": "Book Appointment", "additional_prompt": "Discuss preferred dates and doctors. Show the booking form once agreed.", "entry_behavior": "wait_for_user", "additional_tool_ids": ["show_booking_form", "display_appointment_card"], "position": {"x": 0, "y": 400} } ``` Include `position` (`{x, y}`) on every node so the editor renders cleanly. Start at `y=0`, put `end` at the bottom, and space branches horizontally at `x=-150` and `x=150`; suggested spacing is 200px vertical between levels and 300px horizontal between branches. Keep workflows to 4-7 nodes and always have a path to `end`. Use `entry_behavior` on `override_agent` nodes to choose whether a sub-agent speaks immediately (`generate_immediately`), waits for user input (`wait_for_user`), or lets the platform decide (`auto`). For nested agent transfers, set `enable_nesting` on a `standalone_agent` node and `return_when_nested` on an `end` node that should return control to the parent workflow. ## Procedures Reusable instruction blocks an agent runs when a trigger matches. A procedure is `free_form` (markdown guidance the agent adapts, and the only type that can reference knowledge base documents) or `deterministic` (ordered, typed steps for flows that must run consistently). See [Using the Procedure API](references/using-procedure-api.md) for the full CLI and SDK flow, and [Writing Procedures](references/writing-procedures.md) for the step schema and authoring rules. Procedures live on an agent branch, and every write stages a per-user draft: | Operation | Call | |-----------|------| | List, create, read, update, discard, remove | `/v1/convai/agents/{agent_id}/branches/{branch_id}/procedures...` (`procedures.*` and `procedures.drafts.*` in the SDKs) | | Compile | `POST .../procedures/compile` (`procedures.compile`) | | Publish | `PATCH /v1/convai/agents/{agent_id}?branch_id=...` (`agents.update`) | Semantics worth knowing before writing any of these calls: - Nothing reaches the live agent until you publish. Publishing is not a procedure endpoint; one PATCH on the agent versions every changed procedure draft on the branch. - `GET .../procedures/{procedure_id}` reads branch HEAD and returns `404` until that procedure's first publish. Read the `/draft` variant to see a procedure you just created; do not retry the create. - Compile only when structured (`deterministic`) procedures changed. Compilation turns them into workflow nodes, so the publish must carry the `workflow` that compile returned. Free-form-only changes publish without compiling, because the agent loads free-form procedures from their published versions. - Compile validates structured content and is the only way to check it. On `400` it returns `errors` keyed by procedure ID with the offending field `path`; repair the draft and compile again rather than publishing. - A draft update replaces the whole body. Read the draft first, then resend `name`, `type`, and `trigger` alongside the new `content`. - `content` is markdown for a `free_form` procedure, and a JSON-encoded object with a `trigger` and a `steps` array for a `deterministic` one. Serialize it; do not hand-escape quotes. - Routing is driven by the `trigger` text, not the procedure name. Write concrete, non-overlapping triggers that cover the phrasings a user would actually say. - To restrict the starting agent to selected procedures for one conversation, enable `platform_settings.overrides.enable_procedure_ids_from_client`, then pass their IDs as `procedure_ids` in conversation initiation data. An empty list disables all procedures for that starting agent. - Procedure APIs require `elevenlabs` (Python) or `@elevenlabs/elevenlabs-js` at `2.60.0` or newer. ## Guardrails Layered safety enforcement that runs independently of the LLM — configured under `platform_settings.guardrails`, not in the system prompt. Reference: [Guardrails](https://elevenlabs.io/docs/eleven-agents/best-practices/guardrails). ```json "platform_settings": { "guardrails": { "version": "1", "focus": {"is_enabled": true}, "prompt_injection": {"is_enabled": true}, "content": {"config": {"harassment": {"is_enabled": true, "threshold": 0.5}}}, "custom": { "config": { "configs": [{ "is_enabled": true, "name": "No medical diagnoses", "prompt": "Block the agent from providing medical diagnoses or treatment advice.", "execution_mode": "blocking", "model": "gemini-2.5-flash-lite", "history_message_count": 1, "trigger_action": {"type": "retry", "feedback": "Reason: {{trigger_reason}}"} }] } } } } ``` **Types:** `focus` (on-topic), `prompt_injection` (manipulation defense), `content` (category filters), `custom` (LLM-evaluated domain rules). Content categories include `harassment`, `profanity`, `sexual`, `violence`, `self_harm`, and `medical_and_legal_information` — threshold range `0.0`–`1.0` (default `0.3`). Custom rules use `execution_mode: "blocking"` with a `model`, `history_message_count`, and `trigger_action` (e.g., `retry` with feedback). Custom guardrails evaluate in parallel and fail-open. **Per vertical:** healthcare/finance/legal → enable `medical_and_legal_information`; education/youth → `sexual`/`violence`/`self_harm`/`profanity`; support/sales → `harassment`/`profanity`. All agents benefit from `focus` + `prompt_injection` + 2-4 custom rules. ## Testing Agents Three test types via `POST /v1/convai/agent-testing/create`, then attached with PATCH on the agent. Reference: [Agent Testing](https://elevenlabs.io/docs/eleven-agents/customization/agent-testing). | Type | Purpose | |------|---------| | `llm` | Scenario test — does the agent respond appropriately to a message? | | `tool` | Tool-call test — right tool, right parameters? | | `simulation` | Multi-turn flow with a simulated user persona | ```json // Tool-call test (snake_case throughout; chat_history role is "user" or "agent") { "name": "Books with correct doctor and date", "type": "tool", "chat_history": [ {"role": "user", "message": "Dr. Smith on March 5 at 2pm", "time_in_call_secs": 10} ], "tool_call_parameters": { "referenced_tool": {"id": "show_booking_form", "type": "client"}, "parameters": [ {"path": "doctor_name", "eval": {"type": "llm", "description": "Should reference Dr. Smith"}}, {"path": "date", "eval": {"type": "regex", "pattern": "2025-03-05|March 5"}} ] } } ``` Eval strategies: `exact`, `regex`, `llm`. Prompt evaluation criteria can use binary scoring or numeric scoring with `scoring_mode: "numeric_uniform"`, `max_score`, and `score_instructions`; numeric scores are normalized into the aggregate conversation success percentage. Attach via an agent update: ```bash elevenlabs agents update --agent-id "your-agent-id" \ --json '{"platform_settings": {"testing": {"attached_tests": [{"test_id": "test_xxxx"}]}}}' ``` Run selected tests with `POST /v1/convai/agents/{agent_id}/run-tests`. The request body requires `tests` and accepts `repeat_count` from `1` to `50` for repeated runs. Simulation tests can define up to 30 `success_conditions` prompts; all criteria are evaluated and merged into the final result. Simulation tests can also define `tool_mock_overrides`, keyed by tool ID, to replace shared response mocks for one test. Each override is an array of mocks with a required `mock_result`; set `is_error: true` to exercise a tool-failure path. Overrides only apply to tools enabled for mocking through `tool_mock_config`. For completed conversations, rerun one evaluation criterion with `POST /v1/convai/conversations/{conversation_id}/analysis/evaluations/run` and a request body containing `evaluation_id`. ## Widget Embedding ```html ``` Customize with attributes: `avatar-image-url`, `action-text`, `start-call-text`, `end-call-text`. See [Widget Embedding Reference](references/widget-embedding.md) for all options. ## Outbound Calls Make outbound phone calls using your agent via Twilio or Exotel integration: The examples below use Twilio. See the reference for Exotel usage. ### Python ```python response = client.conversational_ai.twilio.outbound_call( agent_id="your-agent-id", agent_phone_number_id="your-phone-number-id", to_number="+1234567890", call_recording_enabled=True ) print(f"Call initiated: {response.conversation_id}") ``` ### JavaScript ```javascript const response = await client.conversationalAi.twilio.outboundCall({ agentId: "your-agent-id", agentPhoneNumberId: "your-phone-number-id", toNumber: "+1234567890", callRecordingEnabled: true, }); ``` ### CLI ```bash elevenlabs agents twilio outbound_call \ --agent-id "your-agent-id" \ --agent-phone-number-id "your-phone-number-id" \ --to-number "+1234567890" \ --call-recording-enabled true ``` See [Outbound Calls Reference](references/outbound-calls.md) for provider-specific endpoints, configuration overrides, and dynamic variables. ## Managing Agents ### Using CLI (Recommended) ```bash # List agents and check status elevenlabs agents list elevenlabs agents status # Import agents from platform to local config elevenlabs agents pull # Import all agents elevenlabs agents pull --agent # Import specific agent # Push local changes to platform elevenlabs agents push # Upload configurations elevenlabs agents push --dry-run # Preview changes first # Add tools elevenlabs tools add-webhook "Weather API" elevenlabs tools add-client "UI Tool" ``` ### Project Structure The CLI creates a project structure for managing agents: ``` your_project/ ├── agents.json # Agent definitions ├── tools.json # Tool configurations ├── tests.json # Test configurations ├── agent_configs/ # Individual agent configs ├── tool_configs/ # Individual tool configs └── test_configs/ # Individual test configs ``` ### SDK Examples ```python # List agents = client.conversational_ai.agents.list() # Get agent = client.conversational_ai.agents.get(agent_id="your-agent-id") # Update (partial - only include fields to change) client.conversational_ai.agents.update(agent_id="your-agent-id", name="New Name") client.conversational_ai.agents.update(agent_id="your-agent-id", conversation_config={ "agent": {"prompt": {"prompt": "New instructions", "llm": "claude-sonnet-4"}} }) # Delete client.conversational_ai.agents.delete(agent_id="your-agent-id") ``` See [Agent Configuration](references/agent-configuration.md) for all configuration options and SDK examples. ## Error Handling ```python try: agent = client.conversational_ai.agents.create(...) except Exception as e: print(f"API error: {e}") ``` Common errors: **401** (invalid key), **404** (not found), **422** (invalid config), **429** (rate limit) ## References - [Installation Guide](references/installation.md) - SDK setup and migration - [Agent Configuration](references/agent-configuration.md) - All config options and CRUD examples - [Client Tools](references/client-tools.md) - Webhook, client, and system tools - [Using the Procedure API](references/using-procedure-api.md) - Procedure CLI and SDK flow, compile and publish - [Writing Procedures](references/writing-procedures.md) - Trigger and content authoring, step schema - [Widget Embedding](references/widget-embedding.md) - Website integration - [Outbound Calls](references/outbound-calls.md) - Phone call integrations --- ## 2. dubbing - **Source:** https://github.com/elevenlabs/skills/blob/main/dubbing/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/dubbing/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/dubbing.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html Bundled files (2), referenced from this skill's directory: - `references/api-reference.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/dubbing/references/api-reference.md - `references/installation.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/dubbing/references/installation.md --- name: dubbing description: Dub audio and video into other languages using the ElevenLabs Dubbing API (dubbing_v2), preserving the original speakers' voices. Use when translating videos, podcasts, or recordings into other languages, localizing media content, reviewing or correcting dubbing transcripts and translations, or regenerating a dub after edits. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Dubbing Dub audio or video into other languages while preserving the original speakers' voices. Create a project from a file or URL, review and edit the source transcript, add one or more target languages, refine translations per segment, and regenerate outputs. > **Important:** Use the Dubbing Projects API — `elevenlabs.dubbing.project.*` in the SDKs, or the `/v1/dubbing/project` REST endpoints. Do **not** use the legacy v1 dubbing surface (`client.dubbing.create()`, `client.dubbing.get()`, `client.dubbing.audio.get()`, or bare `/v1/dubbing` routes) — that is the older dubbing API, now under Legacy in the API reference. > **Setup:** See [Installation Guide](references/installation.md). The `elevenlabs` CLI and the SDKs read `ELEVENLABS_API_KEY` automatically; REST base URL is `https://api.elevenlabs.io` with your API key in the `xi-api-key` header. ## Concepts | Concept | Meaning | |---------|---------| | **Project** | One source of media (file or URL) plus its source transcript. Prepared (transcribed) once, then rests in `ready` while you add languages. | | **Source transcript** | Editable segments (text, speaker, timing) transcribed from the source. The single source of truth every language is translated from. | | **Language (target)** | One dubbed output language. Each has its own transcript (source segments + a translation per segment) and its own dubbed audio output. | | **Revisions** | Independent monotonic counters. The project's `revision` bumps on source-transcript edits; a language's `revision` bumps on translation edits or source edits that affect it. A language's `output_revision` is the revision its current audio was generated from — when it's behind `revision`, the output is out of date. | **Recommended order of operations:** finalize the source transcript **before** adding any languages. Translations are produced from the source, so correcting the source first means every language starts from the right text — editing the source after a language completes marks it `stale` and requires a (charged) regeneration. > **Enterprise:** Transcript editing and regeneration are available to enterprise workspaces only. Creating projects, adding languages, and downloading dubs work on all plans. ## Workflow 1. **Create** the project from a file or URL → `queued` 2. **Poll** the project until `ready` 3. **Review and finalize the source transcript** (edit/add/delete segments) 4. **Add** one language per target → `queued` → `processing` → `completed` 5. **Download** each language's `outputs.lossless_audio` when `completed` 6. **Refine** translations per segment if needed → the language goes `stale` 7. **Regenerate** the language → `completed` again with fresh output ## Quick Start (Python) ```python import os import time import requests from elevenlabs.client import ElevenLabs elevenlabs = ElevenLabs(api_key=os.getenv("ELEVENLABS_API_KEY")) # 1. Create a project from a local file (or pass source_url=... instead of file) with open("promo.mp4", "rb") as f: project = elevenlabs.dubbing.project.create( file=f, source_language="en", reference="Q3 marketing video", ) # 2. Wait for the source media to be transcribed while True: project = elevenlabs.dubbing.project.get(project.project_id) if project.status == "ready": break if project.status == "failed": raise RuntimeError("Project preparation failed") time.sleep(5) # 3. Add a Spanish language target language = elevenlabs.dubbing.project.language.create( project.project_id, target_language="es", ) # 4. Wait for the dub to finish generating while True: language = elevenlabs.dubbing.project.language.get( project.project_id, language.language_id ) if language.status == "completed": break if language.status == "failed": raise RuntimeError("Dub generation failed") time.sleep(5) # 5. Download the dubbed audio (signed URL, valid ~1 hour — re-fetch the language for a fresh one) audio = requests.get(language.outputs.lossless_audio) with open("promo_es.wav", "wb") as f: f.write(audio.content) ``` ## Quick Start (JavaScript) ```typescript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import { writeFile } from "fs/promises"; const elevenlabs = new ElevenLabsClient(); // 1. Create a project (sourceUrl shown; file upload is also supported) let project = await elevenlabs.dubbing.project.create({ sourceUrl: "https://example.com/promo.mp4", sourceLanguage: "en", reference: "Q3 marketing video", }); // 2. Wait for the source media to be transcribed while (true) { project = await elevenlabs.dubbing.project.get(project.projectId); if (project.status === "ready") break; if (project.status === "failed") throw new Error("Project preparation failed"); await new Promise((resolve) => setTimeout(resolve, 5000)); } // 3. Add a Spanish language target let language = await elevenlabs.dubbing.project.language.create(project.projectId, { targetLanguage: "es", }); // 4. Wait for the dub to finish generating while (true) { language = await elevenlabs.dubbing.project.language.get(project.projectId, language.languageId); if (language.status === "completed") break; if (language.status === "failed") throw new Error("Dub generation failed"); await new Promise((resolve) => setTimeout(resolve, 5000)); } // 5. Download the dubbed audio from the signed URL const response = await fetch(language.outputs!.losslessAudio!); await writeFile("promo_es.wav", Buffer.from(await response.arrayBuffer())); ``` ## Quick Start (CLI) The `elevenlabs` CLI reads `ELEVENLABS_API_KEY` from the environment automatically. ```bash # 1. Create a project (use --source-url "https://..." instead of --file to dub from a URL) elevenlabs dubbing project create --file promo.mp4 --source-language en # → {"project_id": "proj_...", "status": "queued", ...} # 2. Poll until status is "ready" elevenlabs dubbing project get --project-id proj_... # 3. Add a target language elevenlabs dubbing project language create --project-id proj_... --target-language es # 4. Poll the language until "completed", then download outputs.lossless_audio elevenlabs dubbing project language get --project-id proj_... --language-id lang_... ``` ## Create Options `elevenlabs dubbing project create` (REST: `POST /v1/dubbing/project`, `multipart/form-data`) takes **either** `file` **or** `source_url` (not both): | Field | Required | Notes | |-------|----------|-------| | `file` | one of file/source_url | Source media to dub (audio or video), up to 3 GiB | | `source_url` | one of file/source_url | Public URL to fetch the source media from | | `source_language` | no | ISO 639 code (e.g. `en`). Omit to auto-detect — the detected language is reported on the source transcript's `language` field | | `reference` | no | Free-form label to identify the project on your end (max 500 chars) | | `model_id` | no | `dubbing_v2` (default) | | `target_language` | no | Optionally queue the first language target at creation; add more with `language.create` | | `keyterms` | no | Terms to bias transcription/translation toward (product/brand names). Up to 1000 terms; each at most 50 chars and 5 words; `<>{}[]\` not allowed. Repeat the field once per term in multipart | ## Editing the Source Transcript Once the project is `ready`, read the transcript, then correct it before adding languages. Every edit bumps the project's `revision`. Each segment has a stable `id` used to edit or delete it. (Enterprise workspaces only.) ```python # Read the source transcript transcript = elevenlabs.dubbing.project.transcript.get(project_id) # Correct a segment's text — send only the fields to change (text, speaker_id, start_s, end_s) elevenlabs.dubbing.project.transcript.update_segment( project_id, segment_id=transcript.segments[0].id, text="Welcome to our latest product demo.", ) # Add a segment (reuse an existing speaker_id so it's dubbed with that speaker's voice) added = elevenlabs.dubbing.project.transcript.create_segment( project_id, text="Thanks for watching.", speaker_id=transcript.segments[0].speaker_id, start_s=40.0, end_s=42.0, ) # Delete a segment elevenlabs.dubbing.project.transcript.delete_segment(project_id, segment_id=added.segment.id) ``` Via the CLI: `elevenlabs dubbing project transcript get --project-id proj_...`, then update a segment with only the changed fields (`--text`, `--speaker-id`, `--start-s`, `--end-s`): ```bash elevenlabs dubbing project transcript update_segment \ --project-id proj_... --segment-id seg_... \ --text "Welcome to our latest product demo." ``` ## Refining Translations and Regenerating A language's transcript pairs each source segment with its `translation` (`null` = not yet translated; segment ids match the source). Edit a single translation, then regenerate. (Enterprise workspaces only.) ```python # Read the language's translations target = elevenlabs.dubbing.project.language.transcript.get(project_id, language_id) # Refine a single translation (pass translation=None to clear it and mark for re-translation) elevenlabs.dubbing.project.language.transcript.update_segment( project_id, language_id, segment_id=target.segments[0].id, translation="Bienvenido a nuestra última demostración de producto.", ) # Regenerate the dub from the current transcript (charged like a generation) elevenlabs.dubbing.project.language.transcript.regenerate(project_id, language_id) ``` Via the CLI: `elevenlabs dubbing project language transcript update_segment --project-id proj_... --language-id lang_... --segment-id seg_... --translation "..."`, then `elevenlabs dubbing project language transcript regenerate --project-id proj_... --language-id lang_...` (returns `202 Accepted`). A translation edit affects only that language. After the edit, a `completed` language becomes `stale` — it keeps serving its previous output until you regenerate. Poll until `completed`; `output_revision` then equals `revision` and `outputs.lossless_audio` reflects the current transcript. ## Dubbing into Multiple Languages Add one language target per language — each generates independently. Track them all with `language.list` instead of polling one by one: ```python for lang in ["es", "fr", "de", "ja"]: elevenlabs.dubbing.project.language.create(project_id, target_language=lang) while True: result = elevenlabs.dubbing.project.language.list(project_id) if not any(l.status in ("queued", "processing") for l in result.languages): break time.sleep(5) ``` ## States **Project:** | Status | Meaning | |--------|---------| | `queued` | Created; source fetch + preparation enqueued | | `preparing` | Preparation (transcription) running | | `ready` | Source transcript available; add/generate languages. Projects **stay** `ready` — per-language progress lives on the languages | | `failed` | Preparation failed (e.g. source couldn't be fetched or decoded) | **Language:** | Status | Meaning | |--------|---------| | `queued` | Waiting on the project becoming `ready`, or on a generation worker | | `processing` | The dub is being generated | | `completed` | Finished; `outputs` populated with a signed download URL (valid ~1 hour — re-fetch for a fresh one) | | `stale` | Previously completed, but the transcript changed; keeps the last output until regenerated | | `failed` | Generation failed | You can add a language before the project is `ready` — it stays `queued` and starts automatically once the project becomes `ready`. Adding a language accepts optional `model_id` (defaults to the project's) and `voice_settings` (e.g. `{"cloning_strength": 7}`, range 0–10, default 7 — controls how strongly dubbed speakers clone the source voices). ## Error Handling - **401**: Invalid API key - **409 Conflict** on regenerate: The project isn't `ready` or the language isn't settled (e.g. already generating) — wait and retry - **Expired download URL**: `outputs.lossless_audio` is signed and valid ~1 hour; re-fetch the language for a fresh URL - **Transcript editing / regeneration unavailable**: These endpoints are enterprise-only — on other plans, create the project with a finalized source and add languages directly ## References - [Installation Guide](references/installation.md) - [API Reference](references/api-reference.md) — every endpoint with full request/response schemas and SDK method names --- ## 3. music - **Source:** https://github.com/elevenlabs/skills/blob/main/music/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/music/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/music.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html Bundled files (2), referenced from this skill's directory: - `references/api_reference.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/music/references/api_reference.md - `references/installation.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/music/references/installation.md --- name: music description: Generate music using ElevenLabs Music API. Use when creating instrumental tracks, songs with lyrics, background music, jingles, or any AI-generated music composition. Supports prompt-based generation, composition plans for granular control, and detailed output with metadata. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Music Generation Generate music from text prompts - supports instrumental tracks, songs with lyrics, and fine-grained control via composition plans. > **Setup:** See [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only. All examples below default to `music_v2`, the current generation model. Pass `model_id="music_v1"` only when explicitly requested to. ## Quick Start ### Python ```python from elevenlabs import ElevenLabs client = ElevenLabs() audio = client.music.compose( prompt="A chill lo-fi hip hop beat with jazzy piano chords", music_length_ms=30000, model_id="music_v2", ) with open("output.mp3", "wb") as f: for chunk in audio: f.write(chunk) ``` ### TypeScript ```typescript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import { createWriteStream } from "fs"; const client = new ElevenLabsClient(); const audio = await client.music.compose({ prompt: "A chill lo-fi hip hop beat with jazzy piano chords", musicLengthMs: 30000, modelId: "music_v2", }); audio.pipe(createWriteStream("output.mp3")); ``` ### CLI ```bash elevenlabs music compose \ --prompt "A chill lo-fi beat" \ --music-length-ms 30000 \ --model-id music_v2 \ --output output.mp3 ``` ## Methods | Method | Description | |--------|-------------| | `music.compose` | Generate audio from a prompt or composition plan | | `music.stream` | Stream audio chunks as they are generated (paid plans) | | `music.composition_plan.create` | Generate a structured plan for fine-grained control | | `music.compose_detailed` | Generate audio + composition plan + metadata; pass `store_for_inpainting=True` to enable inpainting | | `music.compose_detailed_stream` | Stream audio plus composition plan, metadata, and optional word timestamps as Server-Sent Events | | `music.video_to_music` | Generate background music from one or more uploaded video files | | `music.upload` | Upload an audio file for later inpainting workflows, optionally extracting its composition plan or word-level timestamps | | `music.finetunes.list` | List accessible music finetunes | | `music.finetunes.create` | Train a music finetune from uploaded audio | | `music.finetunes.get` | Retrieve finetune status and metadata | | `music.finetunes.update` | Update finetune metadata or visibility | | `music.finetunes.delete` | Delete a music finetune | See [API Reference](references/api_reference.md) for full parameter details. `music.upload` is available to enterprise clients with access to the inpainting feature. ## Music Finetunes Create a finetune from training audio with [`POST /v1/music/finetunes`](https://elevenlabs.io/docs/api-reference/music/finetunes/create), then poll the [get endpoint](https://elevenlabs.io/docs/api-reference/music/finetunes/get) until its status is `completed`. Pass the returned `id` as `finetune_id` when composing music. Use the [list](https://elevenlabs.io/docs/api-reference/music/finetunes/list), [update](https://elevenlabs.io/docs/api-reference/music/finetunes/update), and [delete](https://elevenlabs.io/docs/api-reference/music/finetunes/delete) endpoints to manage accessible finetunes. ## Video to Music Generate background music from uploaded video clips via [`POST /v1/music/video-to-music`](https://elevenlabs.io/docs/api-reference/music/video-to-music) (`client.music.video_to_music`). This is separate from prompt-based [`music.compose`](https://elevenlabs.io/docs/api-reference/music/compose) (`POST /v1/music`). The API combines videos in order, accepts an optional natural-language description, and lets you steer style with up to 10 tags such as `upbeat` or `cinematic`. This endpoint still defaults to `music_v1`; pass `model_id="music_v2"` to use the newer model. ### Python ```python from elevenlabs import ElevenLabs client = ElevenLabs() audio = client.music.video_to_music( videos=["trailer.mp4"], description="Build suspense, then resolve with a warm cinematic finish.", tags=["cinematic", "suspenseful", "uplifting"], model_id="music_v2", ) with open("video-score.mp3", "wb") as f: for chunk in audio: f.write(chunk) ``` ### TypeScript ```typescript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import { createReadStream, createWriteStream } from "fs"; const client = new ElevenLabsClient(); const audio = await client.music.videoToMusic({ videos: [createReadStream("trailer.mp4")], description: "Build suspense, then resolve with a warm cinematic finish.", tags: ["cinematic", "suspenseful", "uplifting"], modelId: "music_v2", }); audio.pipe(createWriteStream("video-score.mp3")); ``` ### CLI ```bash elevenlabs music video_to_music \ --videos trailer.mp4 \ --description "Build suspense, then resolve with a warm cinematic finish." \ --tags cinematic \ --model-id music_v2 \ --output video-score.mp3 ``` The CLI currently accepts one `--videos` file and one `--tags` value per request; use the Python or TypeScript SDK to send multiple videos or tags. Constraints from the current API schema: - Upload 1-10 video files per request - Keep total combined upload size at or below 200 MB - Keep total combined video duration at or below 600 seconds - Use `description` for high-level musical direction and `tags` for concise style cues ## Composition Plans `music_v2` composition plans are an ordered list of `chunks`. Each chunk specifies its own `text` (section label, lyrics, inline cues), `duration_ms`, `positive_styles`, `negative_styles`, and `context_adherence` (`low`, `medium`, or `high`, default `high`). Up to 30 chunks per plan, each 3,000–120,000 ms, total length 3 s to 10 minutes. Generate a plan first, edit it, then compose: ```python plan = client.music.composition_plan.create( prompt="An epic orchestral piece building to a climax", music_length_ms=60000, model_id="music_v2", ) # Edit chunks in place plan["chunks"][0]["text"] = "[Intro]\nQuiet strings rising" audio = client.music.compose( composition_plan=plan, model_id="music_v2", ) ``` ```typescript const plan = await client.music.compositionPlan.create({ prompt: "An epic orchestral piece building to a climax", musicLengthMs: 60000, modelId: "music_v2", }); plan.chunks[0].text = "[Intro]\nQuiet strings rising"; const audio = await client.music.compose({ compositionPlan: plan, modelId: "music_v2", }); ``` Or hand-build a plan to control lyrics and style per section: ```python composition_plan = { "chunks": [ { "text": "[Verse]\nWalking down an empty street", "duration_ms": 15000, "positive_styles": ["pop", "upbeat", "female vocals", "acoustic guitar"], "negative_styles": ["dark", "slow"], "context_adherence": "high", }, { "text": "[Chorus]\nThis is my moment", "duration_ms": 15000, "positive_styles": ["powerful vocals", "full band"], "negative_styles": [], "context_adherence": "high", }, ] } audio = client.music.compose(composition_plan=composition_plan, model_id="music_v2") ``` ```typescript const compositionPlan = { chunks: [ { text: "[Verse]\nWalking down an empty street", durationMs: 15000, positiveStyles: ["pop", "upbeat", "female vocals", "acoustic guitar"], negativeStyles: ["dark", "slow"], contextAdherence: "high", }, { text: "[Chorus]\nThis is my moment", durationMs: 15000, positiveStyles: ["powerful vocals", "full band"], negativeStyles: [], contextAdherence: "high", }, ], }; const audio = await client.music.compose({ compositionPlan, modelId: "music_v2", }); ``` Put broader characteristics (genre, instrumentation, vocal style) in `positive_styles`, not in `text`. The first chunk's styles set the overall tone — include 6–7 styles there. ## Output Formats Use the `output_format` query parameter on compose, detailed compose, or stream requests to select the generated audio format. `auto` chooses a model-appropriate MP3 format; for `music_v2`, it selects `mp3_48000_192`. Higher-bitrate MP3 options include `mp3_48000_240` and `mp3_48000_320`. ## Streaming For paid plans, stream audio chunks as they are generated instead of waiting for the full file: ```python from io import BytesIO stream = client.music.stream( prompt="A driving synthwave track with arpeggiated leads", music_length_ms=30000, model_id="music_v2", ) buffer = BytesIO() for chunk in stream: if chunk: buffer.write(chunk) ``` ```typescript const stream = await client.music.stream({ prompt: "A driving synthwave track with arpeggiated leads", musicLengthMs: 30000, modelId: "music_v2", }); const chunks: Buffer[] = []; for await (const chunk of stream) { chunks.push(chunk); } ``` ### Detailed streaming Use detailed streaming when the application needs generated music metadata while audio is still arriving. `POST /v1/music/detailed/stream` accepts the same prompt or composition-plan body as detailed compose, streams `text/event-stream`, and can include word timestamps with `with_timestamps`. ```bash elevenlabs music compose_detailed_stream \ --prompt "A bright indie pop hook with warm guitars" \ --music-length-ms 30000 \ --model-id music_v2 \ --with-timestamps true \ --output-format auto ``` ## Inpainting Inpainting edits or extends a stored song by mixing **audio reference chunks** (unchanged slices of a stored song) with new **generation chunks** in a single composition plan. Step 1 — get a `song_id`, either by storing a fresh generation or uploading existing audio: ```python # Option A: keep a generation for later editing result = client.music.compose_detailed( prompt="An upbeat pop song with verse and chorus", music_length_ms=60000, model_id="music_v2", store_for_inpainting=True, ) song_id = result.song_id # Option B: upload an existing track and extract its plan uploaded = client.music.upload( file=open("my-song.mp3", "rb"), extract_composition_plan="music_v2", ) song_id = uploaded.song_id composition_plan = uploaded.composition_plan ``` ```typescript import { createReadStream } from "fs"; // Option A: keep a generation for later editing const result = await client.music.composeDetailed({ prompt: "An upbeat pop song with verse and chorus", musicLengthMs: 60000, modelId: "music_v2", storeForInpainting: true, }); let songId = result.songId; // Option B: upload an existing track and extract its plan const uploaded = await client.music.upload({ file: createReadStream("my-song.mp3"), extractCompositionPlan: "music_v2", }); songId = uploaded.songId; const compositionPlan = uploaded.compositionPlan; ``` Step 2 — compose a plan that references the stored audio and regenerates the part you want to change: ```python plan = { "chunks": [ {"song_id": song_id, "range": {"start_ms": 0, "end_ms": 30000}}, { "text": "[Chorus]\nWe're rising up tonight", "duration_ms": 30000, "positive_styles": ["bigger drums", "layered vocals", "anthemic"], "negative_styles": ["sparse"], "context_adherence": "high", }, ] } audio = client.music.compose(composition_plan=plan, model_id="music_v2") ``` ```typescript const plan = { chunks: [ { songId, range: { startMs: 0, endMs: 30000 } }, { text: "[Chorus]\nWe're rising up tonight", durationMs: 30000, positiveStyles: ["bigger drums", "layered vocals", "anthemic"], negativeStyles: ["sparse"], contextAdherence: "high", }, ], }; const audio = await client.music.compose({ compositionPlan: plan, modelId: "music_v2", }); ``` To match the feel of a stored slice without copying it, attach a `conditioning_ref` (up to 30,000 ms) plus a `condition_strength` of `low`, `medium`, `high`, or `xhigh` to a generation chunk. Conditioning placed on the first chunk influences every later chunk. See [API Reference](references/api_reference.md) for the full inpainting parameter list. ## Content Restrictions - Cannot reference specific artists, bands, or copyrighted lyrics - `bad_prompt` errors include a `prompt_suggestion` with alternative phrasing - `bad_composition_plan` errors include a `composition_plan_suggestion` ## Error Handling ```python try: audio = client.music.compose(prompt="...", music_length_ms=30000) except Exception as e: print(f"API error: {e}") ``` ```typescript try { const audio = await client.music.compose({ prompt: "...", musicLengthMs: 30000, }); } catch (err) { console.error("API error:", err); } ``` Common errors: 401 (invalid key), 422 (invalid params), 429 (rate limit). ## References - [Installation Guide](references/installation.md) - [API Reference](references/api_reference.md) --- ## 4. setup-api-key - **Source:** https://github.com/elevenlabs/skills/blob/main/setup-api-key/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/setup-api-key/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/setup-api-key.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html --- name: setup-api-key description: Guides users through setting up an ElevenLabs API key for REST API and SDK workflows. Use when the user needs to configure an ElevenLabs API key or when an API or SDK request fails because a key is missing. First checks whether ELEVENLABS_API_KEY is already configured and valid, and only runs full setup when needed. license: MIT compatibility: Requires internet access to elevenlabs.io and api.elevenlabs.io. --- # ElevenLabs API Key Setup Guide the user through obtaining and configuring an ElevenLabs API key. ## Workflow ### Step 0: Check for an existing API key first Before asking the user for a key, check for an existing `ELEVENLABS_API_KEY`: 1. Check whether `ELEVENLABS_API_KEY` exists in the current environment. If it does, use that value for this initial check. 2. Only if it is not in the environment, check `.env` for `ELEVENLABS_API_KEY=`. 3. Do not print, quote, or repeat the key. If you mention it, redact it. 4. If an existing key is found, validate it: ```text GET https://api.elevenlabs.io/v1/user Header: xi-api-key: ``` 5. **If existing key validation succeeds:** - Tell the user ElevenLabs is already configured and working - Skip the setup flow - Ask whether they want to replace/rotate the key; if not, stop 6. **If existing key validation fails:** - Tell the user the existing key appears invalid or expired - Continue to Step 1 ### Step 1: Request the API key Tell the user: > To set up ElevenLabs, open the API keys page: https://elevenlabs.io/app/settings/api-keys > > (Need an account? Create one at https://elevenlabs.io/app/sign-up first) > > If you don't have an API key yet: > 1. Click "Create key" > 2. Name it (or use the default) > 3. Set permission for your key. If you provide a key with "User" permission set to "Read" this skill will automatically verify if your key works > 4. Click "Create key" to confirm > 5. **Copy the key immediately** - it's only shown once! > > Do not paste the key into this chat. Instead, copy/paste it into your local `.env` file: > > ``` > ELEVENLABS_API_KEY=your-api-key > ``` > > If `.env` already has an `ELEVENLABS_API_KEY=...` line, replace that line. > Tell me when you've saved it, without sharing the key. Then wait for the user to confirm that the key is saved locally. ### Step 2: Validate and configure After the user says the key is saved: 1. Re-check both `.env` and the current environment for `ELEVENLABS_API_KEY`, but treat `.env` as the source of truth for this step. 2. If `.env` contains a value, validate that value even when the current environment also has a different `ELEVENLABS_API_KEY`. 3. If `.env` does not contain the key: - Tell the user `.env` does not appear to contain `ELEVENLABS_API_KEY`. - Show the expected line again. - If the current environment does contain a key, note that this step still requires saving the key in `.env`. - Remind them not to paste the key into chat. 4. If a `.env` key is found, validate it: ```text GET https://api.elevenlabs.io/v1/user Header: xi-api-key: ``` 5. If validation fails: - Tell the user the local key appears invalid or expired. - Remind them of the API keys page. - Ask them to replace the `.env` value and tell you when it is saved. 6. If validation succeeds, confirm: > Done. ElevenLabs is configured and the key in `.env` works. ## Safety Rules - Never ask the user to paste an API key, token, or secret into chat. - Never print or echo API key values from environment variables or `.env`. - Prefer `.env` or managed secrets over shell history for persistent local configuration. - For browser or client-side apps, keep `ELEVENLABS_API_KEY` on the server and issue short-lived tokens where applicable. --- ## 5. sound-effects - **Source:** https://github.com/elevenlabs/skills/blob/main/sound-effects/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/sound-effects/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/sound-effects.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html Bundled files (1), referenced from this skill's directory: - `references/installation.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/sound-effects/references/installation.md --- name: sound-effects description: Generate sound effects from text descriptions using ElevenLabs. Use when creating sound effects, generating audio textures, producing ambient sounds, cinematic impacts, UI sounds, or any audio that isn't speech. Supports looping, duration control, and prompt influence tuning. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Sound Effects Generate sound effects from text descriptions — supports looping, custom duration, and prompt adherence control. > **Setup:** See [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only. ## Quick Start ### Python ```python from elevenlabs import ElevenLabs client = ElevenLabs() audio = client.text_to_sound_effects.convert( text="Thunder rumbling in the distance with light rain", ) with open("thunder.mp3", "wb") as f: for chunk in audio: f.write(chunk) ``` ### JavaScript ```javascript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import { createWriteStream } from "fs"; const client = new ElevenLabsClient(); const audio = await client.textToSoundEffects.convert({ text: "Thunder rumbling in the distance with light rain", }); audio.pipe(createWriteStream("thunder.mp3")); ``` ### CLI ```bash elevenlabs text-to-sound-effects convert \ --text "Thunder rumbling in the distance with light rain" \ --output thunder.mp3 ``` ## Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `text` | string (required) | — | Description of the desired sound effect | | `model_id` | string | `eleven_text_to_sound_v2` | Model to use | | `duration_seconds` | number \| null | null (auto) | Duration 0.5–30s; auto-calculated if null | | `prompt_influence` | number \| null | 0.3 | How closely to follow the prompt (0–1) | | `loop` | boolean | false | Generate a seamlessly looping sound (v2 model only) | ## Examples with Parameters ```python # Looping ambient sound, 10 seconds audio = client.text_to_sound_effects.convert( text="Gentle forest ambiance with birds chirping", duration_seconds=10.0, prompt_influence=0.5, loop=True, ) # Short UI sound, high prompt adherence audio = client.text_to_sound_effects.convert( text="Soft notification chime", duration_seconds=1.0, prompt_influence=0.8, ) ``` ## Output Formats Pass `--output-format` (CLI) or `output_format` as an SDK parameter: | Format | Description | |--------|-------------| | `mp3_44100_128` | MP3 44.1kHz 128kbps (default) | | `pcm_44100` | Raw uncompressed CD quality | | `opus_48000_128` | Opus 48kHz 128kbps — efficient compressed | | `ulaw_8000` | μ-law 8kHz — telephony | Full list: `mp3_22050_32`, `mp3_24000_48`, `mp3_44100_32`, `mp3_44100_64`, `mp3_44100_96`, `mp3_44100_128`, `mp3_44100_192`, `pcm_8000`, `pcm_16000`, `pcm_22050`, `pcm_24000`, `pcm_32000`, `pcm_44100`, `pcm_48000`, `ulaw_8000`, `alaw_8000`, `opus_48000_32`, `opus_48000_64`, `opus_48000_96`, `opus_48000_128`, `opus_48000_192`. ## Prompt Tips - Be specific: "Heavy rain on a tin roof" > "Rain" - Combine elements: "Footsteps on gravel with distant traffic" - Specify style: "Cinematic braam, horror" or "8-bit retro jump sound" - Mention mood/context: "Eerie wind howling through an abandoned building" ## Error Handling ```python try: audio = client.text_to_sound_effects.convert(text="Explosion") except Exception as e: print(f"API error: {e}") ``` Common errors: - **401**: Invalid API key - **422**: Invalid parameters (check duration range, prompt_influence range) - **429**: Rate limit exceeded ## References - [Installation Guide](references/installation.md) --- ## 6. speech-engine - **Source:** https://github.com/elevenlabs/skills/blob/main/speech-engine/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/speech-engine/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/speech-engine.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html Bundled files (3), referenced from this skill's directory: - `references/installation.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/speech-engine/references/installation.md - `references/javascript-sdk-reference.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/speech-engine/references/javascript-sdk-reference.md - `references/python-sdk-reference.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/speech-engine/references/python-sdk-reference.md --- name: speech-engine description: Add real-time voice conversations to a custom agent runtime with ElevenLabs Speech Engine. Use when building Speech Engine servers, WebSocket handlers, WebRTC browser clients, conversation token endpoints, interruption-aware streaming responses, or voice-enabled chat agents that connect developer-owned server logic to ElevenLabs speech-to-text and text-to-speech. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Speech Engine Add a real-time voice interface to a custom agent. ElevenLabs handles microphone audio, speech-to-text, turn-taking, text-to-speech, and browser playback; your server exposes a Speech Engine WebSocket endpoint and streams response text back. > **Setup:** See [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only. For deeper SDK details, read [JavaScript SDK Reference](references/javascript-sdk-reference.md) or [Python SDK Reference](references/python-sdk-reference.md). ## When to Use Use Speech Engine when the user wants to: - Add voice to an existing chat app or custom server pipeline - Add voice to OpenClaw, Hermes, or a similar agent runtime while keeping agent logic on the developer-owned server - Build a developer-hosted WebSocket server for ElevenLabs voice conversations - Stream response text back as spoken audio after your server validates user intent - Handle user interruptions while a response is still streaming - Build a browser client with `@elevenlabs/react` or `@elevenlabs/client` using a server-issued conversation token Use the `agents` skill instead when the user is creating or configuring a hosted ElevenLabs Conversational AI agent with platform-managed prompts, tools, workflows, phone numbers, or widgets. ## How It Works Each Speech Engine WebSocket connection represents one conversation. 1. The browser sends user audio to ElevenLabs. 2. ElevenLabs sends speech-recognition events to your server. 3. Your server derives trusted application state without letting raw speech text control tools or privileged actions. 4. Your server streams text back through the SDK. 5. ElevenLabs converts the response to speech and plays it in the browser. The SDK manages WebSocket routing, request verification, session lifecycle, ping/pong, turn-taking, and interruption handling. `sendResponse()` / `send_response()` accepts a string or async iterable of response text. Treat speech-recognition text as untrusted user input. Do not map raw speech text directly into model roles, responses, or tool calls. Use deterministic validation, allowlisted intents, or explicit user confirmation before any transcript-derived value affects downstream response or tool logic. ## Implementation Flow 1. Install server dependencies and configure `ELEVENLABS_API_KEY`. 2. Expose your Speech Engine server through a public HTTPS URL for local development, for example with `ngrok http 3001`. 3. Create a Speech Engine resource with `ws_url` / `wsUrl` pointing at the public WebSocket URL, usually `wss://.../ws`. 4. Store the returned Speech Engine ID, for example in `ELEVENLABS_SPEECH_ENGINE_ID`. 5. Start a Speech Engine server with `engine.serve(...)` in Python or `speechEngine.attach(...)` in TypeScript. 6. Issue browser conversation tokens from a server endpoint. Never put `ELEVENLABS_API_KEY` in browser code. 7. Start the client session with `conversationToken`; if the agent should greet first, enable the first-message override on the Speech Engine resource, then set `overrides.agent.firstMessage` in the client. ## Create a Speech Engine ### Python ```python import asyncio import os from dotenv import load_dotenv from elevenlabs import AsyncElevenLabs load_dotenv() elevenlabs = AsyncElevenLabs(api_key=os.getenv("ELEVENLABS_API_KEY")) async def main(): engine = await elevenlabs.speech_engine.create( name="My Speech Engine", speech_engine={"ws_url": os.environ["PUBLIC_WS_URL"]}, overrides={"first_message": True}, ) print(engine.engine_id) asyncio.run(main()) ``` ### TypeScript ```typescript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import "dotenv/config"; const elevenlabs = new ElevenLabsClient({ apiKey: process.env.ELEVENLABS_API_KEY, }); const engine = await elevenlabs.speechEngine.create({ name: "My Speech Engine", speechEngine: { wsUrl: process.env.PUBLIC_WS_URL! }, overrides: { firstMessage: true }, }); console.log(engine.engineId); ``` `PUBLIC_WS_URL` should look like `wss://example.ngrok.app/ws` locally or your production WebSocket route in deployment. The create request can also configure `tts`, `asr`, `turn`, `speech_engine.request_headers` / `speechEngine.requestHeaders`, `overrides`, and `privacy` for custom voices, transcription keywords, turn-taking, server auth headers, client-provided first messages, and recording behavior. See the SDK reference files for expanded examples. ## Server Pattern Run the Speech Engine server at the `ws_url` / `wsUrl` configured on the resource. Keep response generation behind your own validation boundary: raw speech-recognition text should not directly control responses, tools, secrets, or other privileged actions. ### Python ```python engine = await elevenlabs.speech_engine.get(os.environ["ELEVENLABS_SPEECH_ENGINE_ID"]) await engine.serve(port=3001, path="/ws", debug=True, callbacks=validated_callbacks) ``` ### TypeScript ```typescript const engine = await elevenlabs.speechEngine.get(process.env.ELEVENLABS_SPEECH_ENGINE_ID!); engine.attach(httpServer, "/ws", { debug: true, ...validatedCallbacks }); ``` In TypeScript, pass interruption signals to downstream async work when it supports cancellation so interrupted responses stop quickly. In Python, the SDK cancels the previous turn handler when a newer turn arrives. Server callbacks can distinguish clean closes from dropped connections: use `onClose` / `on_close` for clean disconnects and `onDisconnect` / `on_disconnect` for unexpected WebSocket drops. Security note: speech-recognition text can contain prompt-injection attempts from user speech or played audio. Treat it as untrusted input. Convert it into trusted application state before invoking response generation, tools, or privileged workflows. ### Disabling authentication (advanced, dangerous) Both `engine.attach()` (TypeScript) and `engine.serve()` / `SpeechEngineServer` (Python) verify a JWT on every incoming WebSocket by default. This is what proves the connection is really coming from ElevenLabs and not from an attacker who guessed the URL. **Do not turn this off.** An escape hatch exists — `disableAuth: true` in the callback options (TypeScript) or `disable_auth=True` on `serve()` / `SpeechEngineServer(...)` (Python) — for the narrow case where a compensating network-level control is already in place. Without such a control, disabling auth means **any client on the internet that finds your URL can open sessions**. Concretely, an attacker can: - open unlimited conversations to drain your ElevenLabs quota and downstream LLM budget - feed crafted transcripts to your response pipeline, effectively impersonating a user - use your server as an oracle to probe backend state, tools, or prompts Only recommend `disableAuth` / `disable_auth` when the user has already implemented **at least one** of: - **IP allowlist** — the server (or an upstream firewall / load balancer / API gateway) only accepts inbound traffic from [ElevenLabs' documented egress ranges](https://elevenlabs.io/docs/overview/capabilities/speech-engine#ip-allowlisting). - **Custom shared-secret header** — a secret header configured on the Speech Engine resource via `speech_engine.request_headers` / `speechEngine.requestHeaders` at create time, validated by an upstream proxy (or by the developer's own middleware in front of `attach()` / `serve()`) before requests reach the SDK. If the user cannot confirm one of the above is in place, leave the default authentication on. Skipping JWT verification without a mitigation is not an optimization or a convenience — it is unauthenticated public compute. ## Browser Client Create a server-side token endpoint and have the browser request a token before starting the microphone session. Keep the Speech Engine ID and API key on the server. If the client passes `overrides.agent.firstMessage`, the Speech Engine resource must have the first-message override enabled. ```typescript import express from "express"; import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import "dotenv/config"; const app = express(); const elevenlabs = new ElevenLabsClient(); app.get("/api/token", async (_req, res) => { const response = await elevenlabs.conversationalAi.conversations.getWebrtcToken({ agentId: process.env.ELEVENLABS_SPEECH_ENGINE_ID!, }); res.json({ token: response.token }); }); ``` React clients can use `@elevenlabs/react`: ```tsx import { useConversation } from "@elevenlabs/react"; export function VoiceControls() { const conversation = useConversation({ onConnect: () => console.log("connected"), onDisconnect: () => console.log("disconnected"), onError: (error) => console.error(error), }); async function startConversation() { await navigator.mediaDevices.getUserMedia({ audio: true }); const { token } = await fetch("/api/token").then((res) => res.json()); await conversation.startSession({ conversationToken: token, overrides: { agent: { firstMessage: "Hello! How can I help you today?" }, }, }); } return ; } ``` ## References - [Installation Guide](references/installation.md) - [JavaScript SDK Reference](references/javascript-sdk-reference.md) - [Python SDK Reference](references/python-sdk-reference.md) --- ## 7. speech-to-text - **Source:** https://github.com/elevenlabs/skills/blob/main/speech-to-text/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/speech-to-text/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/speech-to-text.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html Bundled files (6), referenced from this skill's directory: - `references/installation.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/speech-to-text/references/installation.md - `references/realtime-client-side.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/speech-to-text/references/realtime-client-side.md - `references/realtime-commit-strategies.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/speech-to-text/references/realtime-commit-strategies.md - `references/realtime-events.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/speech-to-text/references/realtime-events.md - `references/realtime-server-side.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/speech-to-text/references/realtime-server-side.md - `references/transcription-options.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/speech-to-text/references/transcription-options.md --- name: speech-to-text description: Transcribe audio to text using ElevenLabs Scribe v2. Use when converting audio/video to text, generating subtitles, transcribing meetings, or processing spoken content. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Speech-to-Text Transcribe audio to text with Scribe v2 - supports 90+ languages, speaker diarization, and word-level timestamps. > **Setup:** See [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only. ## Quick Start ### Python ```python from elevenlabs import ElevenLabs client = ElevenLabs() with open("audio.mp3", "rb") as audio_file: result = client.speech_to_text.convert(file=audio_file, model_id="scribe_v2") print(result.text) ``` ### JavaScript ```javascript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import { createReadStream } from "fs"; const client = new ElevenLabsClient(); const result = await client.speechToText.convert({ file: createReadStream("audio.mp3"), modelId: "scribe_v2", }); console.log(result.text); ``` ### CLI ```bash elevenlabs speech-to-text convert --file audio.mp3 --model-id scribe_v2 ``` ## Models | Model ID | Description | Best For | |----------|-------------|----------| | `scribe_v2` | State-of-the-art accuracy, 90+ languages | Batch transcription, subtitles, long-form audio | | `scribe_v2_realtime` | Low latency (~150ms) | Live transcription, voice agents | | `scribe_v2_realtime_turbo` | Realtime transcription variant | Live transcription | | `scribe_v2_realtime_lite` | Realtime transcription variant | Live transcription | ## Transcription with Timestamps Word-level timestamps include type classification and speaker identification: ```python result = client.speech_to_text.convert( file=audio_file, model_id="scribe_v2", timestamps_granularity="word" ) for word in result.words: print(f"{word.text}: {word.start}s - {word.end}s (type: {word.type})") ``` ## Speaker Diarization Identify WHO said WHAT - the model labels each word with a speaker ID, useful for meetings, interviews, or any multi-speaker audio: ```python result = client.speech_to_text.convert( file=audio_file, model_id="scribe_v2", diarize=True ) for word in result.words: print(f"[{word.speaker_id}] {word.text}") ``` For call recordings, the batch API can label diarized speakers as `agent` and `customer` by setting `detect_speaker_roles=true` alongside `diarize=true`. This option is not compatible with `use_multi_channel=true`. If your workspace has registered speaker profiles, set `use_speaker_library=true` with `diarize=true` to match detected speakers against the speaker library. ```bash elevenlabs speech-to-text convert \ --file call.mp3 \ --model-id scribe_v2 \ --diarize true \ --detect-speaker-roles true \ --use-speaker-library true ``` ## Multichannel Audio Use `use_multi_channel=true` when each speaker is isolated on a separate audio channel. By default, the API returns one transcript per channel under `transcripts`; set `multichannel_output_style="combined"` to receive one transcript merged by timestamp, with `channel_index` on each word. ```python result = client.speech_to_text.convert( file=audio_file, model_id="scribe_v2", use_multi_channel=True, multichannel_output_style="combined", ) ``` ## Keyterm Prompting Help the model recognize specific words it might otherwise mishear - product names, technical jargon, or unusual spellings (up to 100 terms): ```python result = client.speech_to_text.convert( file=audio_file, model_id="scribe_v2", keyterms=["ElevenLabs", "Scribe", "API"] ) ``` ## Language Detection Automatic detection with optional language hint: ```python result = client.speech_to_text.convert( file=audio_file, model_id="scribe_v2", language_code="eng" # ISO 639-1 or ISO 639-3 code ) print(f"Detected: {result.language_code} ({result.language_probability:.0%})") ``` ## Supported Formats **Audio:** MP3, WAV, M4A, FLAC, OGG, WebM, AAC, AIFF, Opus **Video:** MP4, AVI, MKV, MOV, WMV, FLV, WebM, MPEG, 3GPP **Limits:** Up to 5.0GB file size, 10 hours duration ## Response Format ```json { "text": "The full transcription text", "language_code": "eng", "language_probability": 0.98, "words": [ {"text": "The", "start": 0.0, "end": 0.15, "type": "word", "speaker_id": "speaker_0"}, {"text": " ", "start": 0.15, "end": 0.16, "type": "spacing", "speaker_id": "speaker_0"} ] } ``` **Word types:** - `word` - An actual spoken word - `spacing` - Whitespace between words (useful for precise timing) - `audio_event` - Non-speech sounds the model detected (laughter, applause, music, etc.) ## Error Handling ```python try: result = client.speech_to_text.convert(file=audio_file, model_id="scribe_v2") except Exception as e: print(f"Transcription failed: {e}") ``` Common errors: - **401**: Invalid API key - **422**: Invalid parameters - **429**: Rate limit exceeded ## Tracking Costs Monitor usage via `request-id` response header: ```python response = client.speech_to_text.with_raw_response.convert(file=audio_file, model_id="scribe_v2") result = response.data print(f"Request ID: {response.headers.get('request-id')}") ``` ## Real-Time Streaming For live transcription with ultra-low latency (~150ms), use the real-time API. The real-time API produces two types of transcripts: - **Partial transcripts**: Interim results that update frequently as audio is processed - use these for live feedback (e.g., showing text as the user speaks) - **Committed transcripts**: Final, stable results after you "commit" - use these as the source of truth for your application A "commit" tells the model to finalize the current segment. You can commit manually (e.g., when the user pauses) or use Voice Activity Detection (VAD) to auto-commit on silence. ### Python (Server-Side) ```python import asyncio from elevenlabs import ElevenLabs client = ElevenLabs() async def transcribe_realtime(): async with client.speech_to_text.realtime.connect( model_id="scribe_v2_realtime", include_timestamps=True, keyterms=["ElevenLabs", "Scribe"], no_verbatim=True, ) as connection: await connection.stream_url("https://example.com/audio.mp3") async for event in connection: if event.type == "partial_transcript": print(f"Partial: {event.text}") elif event.type == "committed_transcript": print(f"Final: {event.text}") asyncio.run(transcribe_realtime()) ``` ### JavaScript (Client-Side with React) ```typescript import { useScribe, CommitStrategy } from "@elevenlabs/react"; function TranscriptionComponent() { const [transcript, setTranscript] = useState(""); const scribe = useScribe({ modelId: "scribe_v2_realtime", commitStrategy: CommitStrategy.VAD, // Auto-commit on silence for mic input keyterms: ["ElevenLabs", "Scribe"], noVerbatim: true, includeLanguageDetection: true, onPartialTranscript: (data) => console.log("Partial:", data.text), onCommittedTranscript: (data) => setTranscript((prev) => prev + data.text), }); const start = async () => { // Get token from your backend (never expose API key to client) const { token } = await fetch("/scribe-token").then((r) => r.json()); await scribe.connect({ token, microphone: { echoCancellation: true, noiseSuppression: true }, }); }; return ; } ``` ### Commit Strategies | Strategy | Description | |----------|-------------| | **Manual** | You call `commit()` when ready - use for file processing or when you control the audio segments | | **VAD** | Voice Activity Detection auto-commits when silence is detected - use for live microphone input | Set `includeLanguageDetection: true` to receive the detected language code in delayed final transcript events. ```typescript // React: set commitStrategy on the hook (recommended for mic input) import { useScribe, CommitStrategy } from "@elevenlabs/react"; const scribe = useScribe({ modelId: "scribe_v2_realtime", commitStrategy: CommitStrategy.VAD, keyterms: ["ElevenLabs", "Scribe"], noVerbatim: true, // Optional VAD tuning: vadSilenceThresholdSecs: 1.5, vadThreshold: 0.4, }); ``` ```javascript // JavaScript client: pass vad config on connect const connection = await client.speechToText.realtime.connect({ modelId: "scribe_v2_realtime", keyterms: ["ElevenLabs", "Scribe"], noVerbatim: true, vad: { silenceThresholdSecs: 1.5, threshold: 0.4, }, }); ``` ### Event Types | Event | Description | |-------|-------------| | `partial_transcript` | Live interim results | | `final_transcript` | Stable segment result sent before the segment is committed | | `final_transcript_with_timestamps` | Delayed final result with timestamps and/or detected language | | `committed_transcript` | Final results after commit | | `committed_transcript_with_timestamps` | Final with word timing | | `committed_transcript_entities` | Entities detected in a committed segment | | `invalid_request` | Connection parameters were rejected and the session closes | | `error` | Error occurred | See real-time references for complete documentation. ## References - [Installation Guide](references/installation.md) - [Transcription Options](references/transcription-options.md) - [Real-Time Client-Side Streaming](references/realtime-client-side.md) - [Real-Time Server-Side Streaming](references/realtime-server-side.md) - [Commit Strategies](references/realtime-commit-strategies.md) - [Real-Time Event Reference](references/realtime-events.md) --- ## 8. text-to-speech - **Source:** https://github.com/elevenlabs/skills/blob/main/text-to-speech/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/text-to-speech/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/text-to-speech.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html Bundled files (3), referenced from this skill's directory: - `references/installation.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/text-to-speech/references/installation.md - `references/streaming.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/text-to-speech/references/streaming.md - `references/voice-settings.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/text-to-speech/references/voice-settings.md --- name: text-to-speech description: Convert text to speech using ElevenLabs voice AI. Use when generating audio from text, creating voiceovers, building voice apps, or synthesizing speech in 70+ languages. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Text-to-Speech Generate natural speech from text - supports 70+ languages, multiple models for quality vs latency tradeoffs. > **Setup:** See [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only. ## Quick Start ### Python ```python from elevenlabs import ElevenLabs client = ElevenLabs() audio = client.text_to_speech.convert( text="Hello, welcome to ElevenLabs!", voice_id="JBFqnCBsd6RMkjVDRZzb", # George model_id="eleven_multilingual_v2" ) with open("output.mp3", "wb") as f: for chunk in audio: f.write(chunk) ``` ### JavaScript ```javascript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import { createWriteStream } from "fs"; import { Readable } from "stream"; const client = new ElevenLabsClient(); const audio = await client.textToSpeech.convert("JBFqnCBsd6RMkjVDRZzb", { text: "Hello, welcome to ElevenLabs!", modelId: "eleven_multilingual_v2", }); // convert() returns a web ReadableStream — bridge it to a Node stream to write to disk Readable.fromWeb(audio).pipe(createWriteStream("output.mp3")); ``` ### CLI Use `say` to play text immediately with the default voice and `eleven_v3` model: ```bash elevenlabs say "Hello!" ``` Pipe text into `say` when another command produces the input: ```bash echo "The build finished successfully." | elevenlabs say ``` Use the API command when you need to set request parameters directly: ```bash elevenlabs text-to-speech convert --voice-id JBFqnCBsd6RMkjVDRZzb \ --text "Hello!" --model-id eleven_multilingual_v2 --output output.mp3 ``` The CLI reads `ELEVENLABS_API_KEY` from the environment automatically. ## Models | Model ID | Languages | Latency | Best For | |----------|-----------|---------|----------| | `eleven_v3` | 70+ | Standard | Highest quality, emotional range | | `eleven_multilingual_v2` | 29 | Standard | High quality, long-form content | | `eleven_flash_v2_5` | 32 | ~75ms | Ultra-low latency, real-time | | `eleven_flash_v2` | English | ~75ms | English-only, fastest | | `eleven_turbo_v2_5` | 32 | ~250-300ms | Balanced quality/speed | | `eleven_turbo_v2` | English | ~250-300ms | English-only, balanced | ## Voice IDs Use pre-made voices or create custom voices in the dashboard. **Popular voices:** - `JBFqnCBsd6RMkjVDRZzb` - George (male, narrative) - `EXAVITQu4vr4xnSDxMaL` - Sarah (female, soft) - `onwK4e9ZLuTAKqWW03F9` - Daniel (male, authoritative) - `XB0fDUnXU5powFXDhCwa` - Charlotte (female, conversational) ```python voices = client.voices.get_all() for voice in voices.voices: print(f"{voice.voice_id}: {voice.name}") ``` ## Voice Settings Fine-tune how the voice sounds: - **Stability**: How consistent the voice stays. Lower values = more emotional range and variation, but can sound unstable. Higher = steady, predictable delivery. - **Similarity boost**: How closely to match the original voice sample. Higher values sound more like the original but may amplify audio artifacts. - **Style**: Exaggerates the voice's unique style characteristics (only works with v2+ models). - **Speaker boost**: Post-processing that enhances clarity and voice similarity. ```python from elevenlabs import VoiceSettings audio = client.text_to_speech.convert( text="Customize my voice settings.", voice_id="JBFqnCBsd6RMkjVDRZzb", voice_settings=VoiceSettings( stability=0.5, similarity_boost=0.75, style=0.5, speed=1.0, # 0.25 to 4.0 (default 1.0) use_speaker_boost=True ) ) ``` ## Language Selection Use `language_code` with models that support language enforcement to guide pronunciation and text normalization. Unsupported language codes are ignored, and `language_code` is not supported on `eleven_multilingual_v2`. ```python audio = client.text_to_speech.convert( text="Bonjour, comment allez-vous?", voice_id="JBFqnCBsd6RMkjVDRZzb", model_id="eleven_v3", language_code="fr" # ISO 639-1 code ) ``` ## Text Normalization Controls how numbers, dates, and abbreviations are converted to spoken words. For example, "01/15/2026" becomes "January fifteenth, twenty twenty-six": - `"auto"` (default): Model decides based on context - `"on"`: Always normalize (use when you want natural speech) - `"off"`: Speak literally (use when you want "zero one slash one five...") ```python audio = client.text_to_speech.convert( text="Call 1-800-555-0123 on 01/15/2026", voice_id="JBFqnCBsd6RMkjVDRZzb", apply_text_normalization="on" ) ``` ## Request Stitching When generating long audio in multiple requests, the audio can have pops, unnatural pauses, or tone shifts at the boundaries. Request stitching solves this by letting each request know what comes before/after it: ```python # First request audio1 = client.text_to_speech.convert( text="This is the first part.", voice_id="JBFqnCBsd6RMkjVDRZzb", next_text="And this continues the story." ) # Second request using previous context audio2 = client.text_to_speech.convert( text="And this continues the story.", voice_id="JBFqnCBsd6RMkjVDRZzb", previous_text="This is the first part." ) ``` ## Output Formats | Format | Description | |--------|-------------| | `mp3_44100_128` | MP3 44.1kHz 128kbps (default) - compressed, good for web/apps | | `mp3_44100_192` | MP3 44.1kHz 192kbps (Creator+) - higher quality compressed | | `mp3_44100_64` | MP3 44.1kHz 64kbps - lower quality, smaller files | | `mp3_22050_32` | MP3 22.05kHz 32kbps - smallest MP3 files | | `pcm_16000` | Raw PCM 16kHz - use for real-time processing | | `pcm_22050` | Raw PCM 22.05kHz | | `pcm_24000` | Raw PCM 24kHz - good balance for streaming | | `pcm_44100` | Raw PCM 44.1kHz (Pro+) - CD quality | | `pcm_48000` | Raw PCM 48kHz (Pro+) - highest quality | | `ulaw_8000` | μ-law 8kHz - standard for phone systems (Twilio, telephony) | | `alaw_8000` | A-law 8kHz - telephony (alternative to μ-law) | | `opus_48000_64` | Opus 48kHz 64kbps - efficient streaming codec | | `wav_44100` | WAV 44.1kHz - uncompressed with headers | ## Streaming For real-time applications, use the `stream` method (returns audio chunks as they're generated): ```python audio_stream = client.text_to_speech.stream( text="This text will be streamed as audio.", voice_id="JBFqnCBsd6RMkjVDRZzb", model_id="eleven_flash_v2_5" # Ultra-low latency ) for chunk in audio_stream: play_audio(chunk) ``` See [references/streaming.md](references/streaming.md) for WebSocket streaming. ## Error Handling ```python try: audio = client.text_to_speech.convert( text="Generate speech", voice_id="invalid-voice-id" ) except Exception as e: print(f"API error: {e}") ``` Common errors: - **401**: Invalid API key - **422**: Invalid parameters (check voice_id, model_id) - **429**: Rate limit exceeded ## Tracking Costs Monitor character usage via response headers (`x-character-count`, `request-id`): ```python response = client.text_to_speech.convert.with_raw_response( text="Hello!", voice_id="JBFqnCBsd6RMkjVDRZzb", model_id="eleven_multilingual_v2" ) audio = response.parse() print(f"Characters used: {response.headers.get('x-character-count')}") ``` ## References - [Installation Guide](references/installation.md) - [Streaming Audio](references/streaming.md) - [Voice Settings](references/voice-settings.md) --- ## 9. voice-changer - **Source:** https://github.com/elevenlabs/skills/blob/main/voice-changer/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/voice-changer/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/voice-changer.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html Bundled files (1), referenced from this skill's directory: - `references/installation.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/voice-changer/references/installation.md --- name: voice-changer description: Transform the voice in an audio recording into a different target voice while preserving emotion, timing, and delivery using the ElevenLabs Voice Changer (speech-to-speech) API. Use when converting one voice to another, changing the speaker/narrator of an existing recording, dubbing a voice-over in a different voice, creating character voices from a scratch performance, anonymizing a speaker, or any "voice conversion / voice transfer / speech-to-speech" task. Make sure to use this skill whenever the user mentions voice changing, voice conversion, speech-to-speech, swapping a voice in audio, re-voicing a clip, or applying a different voice to an existing recording — even if they don't explicitly say "voice changer". Do not use this skill to create or clone a new voice from a voice sample — that is voice cloning (IVC/PVC), a separate feature; this skill only converts an existing recording into an existing voice_id. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Voice Changer Transform the voice in an audio recording into a different target voice. Voice Changer (previously called Speech-to-Speech — the API endpoint and SDK methods still use the `speech_to_speech` / `speechToSpeech` name) keeps the original performance — emotion, pacing, intonation, breaths, whispers, laughs, cries — and only swaps who is speaking. > **Setup:** See [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only. ## Key Facts - **Maximum input length:** 5 minutes per request — split longer recordings into chunks and stitch the outputs. - **Maximum file size:** 50 MB per request — compress to MP3 if your source is larger. - **Pricing:** 1,000 characters per minute of audio processed (duration-based, not text-based). - **Recommended model:** `eleven_multilingual_sts_v2` — often outperforms `eleven_english_sts_v2` even for English-only content. ## Quick Start ### Python ```python from elevenlabs import ElevenLabs client = ElevenLabs() with open("source.mp3", "rb") as audio_file: audio_stream = client.speech_to_speech.convert( voice_id="JBFqnCBsd6RMkjVDRZzb", # George audio=audio_file, model_id="eleven_multilingual_sts_v2", output_format="mp3_44100_128", ) with open("converted.mp3", "wb") as f: for chunk in audio_stream: f.write(chunk) ``` ### JavaScript ```javascript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import { createReadStream, createWriteStream } from "fs"; const client = new ElevenLabsClient(); const audioStream = await client.speechToSpeech.convert("JBFqnCBsd6RMkjVDRZzb", { audio: createReadStream("source.mp3"), modelId: "eleven_multilingual_sts_v2", outputFormat: "mp3_44100_128", }); audioStream.pipe(createWriteStream("converted.mp3")); ``` ### CLI ```bash elevenlabs speech-to-speech convert \ --voice-id JBFqnCBsd6RMkjVDRZzb \ --audio source.mp3 \ --model-id eleven_multilingual_sts_v2 \ --output-format mp3_44100_128 \ --output converted.mp3 ``` ## Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `voice_id` | string (required) | — | Target voice to speak in. Use a pre-made voice ID, a cloned voice, or a voice from the library | | `audio` | file (required) | — | Source audio whose performance (emotion, timing, delivery) will be preserved | | `model_id` | string | `eleven_english_sts_v2` | `eleven_multilingual_sts_v2` for 29 languages, `eleven_english_sts_v2` for English-only | | `output_format` | string | `mp3_44100_128` | See output formats table below | | `voice_settings` | JSON string | — | Override stored voice settings for this request only | | `seed` | integer | — | Best-effort deterministic sampling (0 – 4294967295) | | `remove_background_noise` | boolean | `false` | Run the isolation model on the input before conversion | | `file_format` | string | `other` | `other` for any encoded audio, or `pcm_s16le_16` for 16-bit PCM mono @ 16kHz little-endian (lower latency) | | `optimize_streaming_latency` | int (query) | — | 0–4. Trade quality for latency. `4` is fastest but disables the text normalizer | | `enable_logging` | boolean (query) | `true` | Set to `false` for zero-retention mode (enterprise only — disables history/stitching) | ## Models | Model ID | Languages | Best For | |----------|-----------|----------| | `eleven_multilingual_sts_v2` | 29 | Recommended for everything — often outperforms the English model even on English audio | | `eleven_english_sts_v2` | English | API default — English-only fallback | Only models whose `can_do_voice_conversion` property is true can be used here. Voice Changer does not currently have a low-latency "flash/turbo" tier — if you need one, keep `pcm_s16le_16` input, an `opus_*` / low-bitrate `mp3_*` output, and raise `optimize_streaming_latency`. ### Languages (`eleven_multilingual_sts_v2`) English (US, UK, AU, CA), Japanese, Chinese, German, Hindi, French (FR, CA), Korean, Portuguese (BR, PT), Italian, Spanish (ES, MX), Indonesian, Dutch, Turkish, Filipino, Polish, Swedish, Bulgarian, Romanian, Arabic (SA, AE), Czech, Greek, Finnish, Croatian, Malay, Slovak, Danish, Tamil, Ukrainian, Russian. ## Target Voices Use any voice ID from pre-made voices, your cloned voices, or the voice library. **Popular voices:** - `JBFqnCBsd6RMkjVDRZzb` — George (male, narrative) - `EXAVITQu4vr4xnSDxMaL` — Sarah (female, soft) - `onwK4e9ZLuTAKqWW03F9` — Daniel (male, authoritative) - `XB0fDUnXU5powFXDhCwa` — Charlotte (female, conversational) ```python voices = client.voices.get_all() for voice in voices.voices: print(f"{voice.voice_id}: {voice.name}") ``` ## Converting from a URL ```python import requests from io import BytesIO from elevenlabs import ElevenLabs client = ElevenLabs() audio_url = "https://storage.googleapis.com/eleven-public-cdn/audio/marketing/nicole.mp3" response = requests.get(audio_url) audio_data = BytesIO(response.content) audio_stream = client.speech_to_speech.convert( voice_id="JBFqnCBsd6RMkjVDRZzb", audio=audio_data, model_id="eleven_multilingual_sts_v2", output_format="mp3_44100_128", ) with open("converted.mp3", "wb") as f: for chunk in audio_stream: f.write(chunk) ``` ## Voice Settings Override Fine-tune the target voice for a single request without changing its stored defaults: ```python from elevenlabs import VoiceSettings audio_stream = client.speech_to_speech.convert( voice_id="JBFqnCBsd6RMkjVDRZzb", audio=audio_file, model_id="eleven_multilingual_sts_v2", voice_settings=VoiceSettings( stability=0.5, similarity_boost=0.75, style=0.0, use_speaker_boost=True, ), ) ``` - **Stability**: lower = more emotional range (follows the source more freely), higher = steadier delivery. - **Similarity boost**: higher = closer to the target voice's timbre, may amplify source artifacts. - **Style**: exaggerates the target voice's unique characteristics (v2+ models). - **Speaker boost**: post-processing to sharpen clarity of the target voice. ## Cleaning Up Noisy Source Audio If the input recording is noisy, either pre-process with the voice-isolator skill or pass `remove_background_noise=True` to do it in a single call: ```python audio_stream = client.speech_to_speech.convert( voice_id="JBFqnCBsd6RMkjVDRZzb", audio=audio_file, model_id="eleven_multilingual_sts_v2", remove_background_noise=True, ) ``` Cleaner input almost always produces better conversion — the model is trying to match phonemes and prosody, and background noise gets in the way. ## Low-Latency PCM Input If you already have raw 16-bit PCM mono @ 16kHz, passing `file_format="pcm_s16le_16"` skips decoding and reduces latency: ```python audio_stream = client.speech_to_speech.convert( voice_id="JBFqnCBsd6RMkjVDRZzb", audio=pcm_bytes, model_id="eleven_multilingual_sts_v2", file_format="pcm_s16le_16", ) ``` Pair this with `optimize_streaming_latency` (0–4) as a query param for further latency reductions at some quality cost. ## Output Formats | Format | Description | |--------|-------------| | `mp3_44100_128` | MP3 44.1kHz 128kbps (default) — good for web/apps | | `mp3_44100_192` | MP3 44.1kHz 192kbps (Creator+) — higher quality | | `mp3_44100_64` | MP3 44.1kHz 64kbps — smaller files | | `mp3_22050_32` | MP3 22.05kHz 32kbps — smallest MP3 | | `pcm_16000` | Raw PCM 16kHz — real-time pipelines | | `pcm_24000` | Raw PCM 24kHz — good streaming balance | | `pcm_44100` | Raw PCM 44.1kHz (Pro+) — CD quality | | `pcm_48000` | Raw PCM 48kHz (Pro+) — highest quality | | `ulaw_8000` | μ-law 8kHz — Twilio / telephony | | `alaw_8000` | A-law 8kHz — telephony | | `opus_48000_64` | Opus 48kHz 64kbps — efficient streaming | ## Deterministic Output Pass a `seed` to make repeated conversions of the same input return (best-effort) identical audio — useful for testing and A/B comparisons. ```python audio_stream = client.speech_to_speech.convert( voice_id="JBFqnCBsd6RMkjVDRZzb", audio=audio_file, model_id="eleven_multilingual_sts_v2", seed=12345, ) ``` ## Input Audio Best Practices The conversion quality is bounded by the input recording — the model can only swap the timbre, not rescue a bad source. A few practical rules: - **Be expressive.** Whisper, shout, laugh, cry — the model preserves all of it. Flat input gives you flat output. - **Watch microphone gain.** Too quiet and the model under-detects phonemes; too loud and clipping bleeds into the conversion. Aim for healthy peaks, no clipping. - **Accent and cadence transfer from the source, not the target.** If you read in an American accent and target the British "George" voice, you get George's timbre with an American accent. To dub *into* a different accent or language, record someone speaking in that target accent/language and convert into a cloned/library voice. - **Clean up noise first.** Either pass `remove_background_noise=True` or run the source through the voice-isolator skill before conversion. Noise hurts more here than in TTS. - **Split long recordings.** Anything over 5 minutes must be chunked. Cut at natural pauses, convert each piece, and concatenate the resulting audio. ## Common Workflows - **Re-voice a narration** — keep the performance of a scratch recording, swap in a different narrator voice. - **Localize / dub** — convert a voice-over into the same speaker's cloned voice in another language (using `eleven_multilingual_sts_v2`). - **Create character voices** — act out a line yourself, convert into a distinctive character voice for games or animation. - **Anonymize a speaker** — replace a recognizable voice with a neutral pre-made voice while preserving what was said and how. - **Pair with voice-isolator** — isolate the source voice first (or set `remove_background_noise=True`) for noisy field recordings before conversion. - **Pair with voice cloning** — clone a target voice from a short sample, then use its `voice_id` here as the conversion target. ## Error Handling ```python try: audio_stream = client.speech_to_speech.convert( voice_id="JBFqnCBsd6RMkjVDRZzb", audio=audio_file, model_id="eleven_multilingual_sts_v2", ) except Exception as e: print(f"Voice changer failed: {e}") ``` Common errors: - **401**: Invalid API key - **422**: Invalid parameters (check `voice_id`, `model_id`, or `file_format` vs the supplied audio) - **429**: Rate limit exceeded ## References - [Installation Guide](references/installation.md) --- ## 10. voice-isolator - **Source:** https://github.com/elevenlabs/skills/blob/main/voice-isolator/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/voice-isolator/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/voice-isolator.md - **Licence:** MIT — https://spdx.org/licenses/MIT.html Bundled files (1), referenced from this skill's directory: - `references/installation.md` — https://raw.githubusercontent.com/elevenlabs/skills/main/voice-isolator/references/installation.md --- name: voice-isolator description: Remove background noise and isolate vocals/speech from audio using ElevenLabs Voice Isolator (audio isolation) API. Use when cleaning up noisy recordings, removing music or background ambience from dialogue, isolating speech from field recordings, preparing audio for transcription, extracting vocals, or any "denoise / clean up / isolate voice" task. license: MIT compatibility: Requires internet access and an ElevenLabs API key (ELEVENLABS_API_KEY). metadata: {"openclaw": {"requires": {"env": ["ELEVENLABS_API_KEY"]}, "primaryEnv": "ELEVENLABS_API_KEY"}} --- # ElevenLabs Voice Isolator Removes background noise from audio and isolates vocals/speech — useful for cleaning up noisy recordings, prepping audio for transcription, or pulling dialogue out of a mixed track. > **Setup:** See [Installation Guide](references/installation.md). For JavaScript, use `@elevenlabs/*` packages only. ## Quick Start ### Python ```python from elevenlabs import ElevenLabs client = ElevenLabs() with open("noisy.mp3", "rb") as audio_file: audio_stream = client.audio_isolation.convert(audio=audio_file) with open("clean.mp3", "wb") as f: for chunk in audio_stream: f.write(chunk) ``` ### JavaScript ```javascript import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js"; import { createReadStream, createWriteStream } from "fs"; const client = new ElevenLabsClient(); const audioStream = await client.audioIsolation.convert({ audio: createReadStream("noisy.mp3"), }); audioStream.pipe(createWriteStream("clean.mp3")); ``` ### CLI ```bash elevenlabs audio-isolation convert --audio noisy.mp3 --output clean.mp3 ``` ## Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `audio` | file (required) | — | Audio file with vocals/speech to isolate | | `file_format` | string | `other` | `other` for any encoded audio, or `pcm_s16le_16` for 16-bit PCM mono @ 16kHz little-endian (lower latency) | ## Isolating from a URL ```python import requests from io import BytesIO from elevenlabs import ElevenLabs client = ElevenLabs() audio_url = "https://example.com/noisy.mp3" response = requests.get(audio_url) audio_data = BytesIO(response.content) audio_stream = client.audio_isolation.convert(audio=audio_data) with open("clean.mp3", "wb") as f: for chunk in audio_stream: f.write(chunk) ``` ## Low-Latency PCM Input If you already have raw 16-bit PCM mono @ 16kHz, passing `file_format="pcm_s16le_16"` skips decoding and reduces latency: ```python audio_stream = client.audio_isolation.convert( audio=pcm_bytes, file_format="pcm_s16le_16", ) ``` ## Supported Formats Any common encoded audio/video container works as input (MP3, WAV, M4A, FLAC, OGG, WebM, MP4, etc.). Response is a streamed MP3 by default. ## Common Workflows - **Clean up interview/podcast recordings** — strip room tone, HVAC, traffic before editing. - **Prep noisy audio for Speech-to-Text** — isolate voice first, then pass through `speech_to_text.convert()` for better transcription accuracy. - **Extract dialogue from mixed tracks** — pull vocals out of a track with music/SFX. - **Pre-processing for Voice Changer** — isolate the source voice before applying voice transformation. ## Error Handling ```python try: audio_stream = client.audio_isolation.convert(audio=audio_file) except Exception as e: print(f"Voice isolation failed: {e}") ``` Common errors: - **401**: Invalid API key - **422**: Invalid parameters (e.g. wrong `file_format` for the supplied audio) - **429**: Rate limit exceeded ## References - [Installation Guide](references/installation.md) --- ## Part: Credited skills --- ## 11. update-skills-from-changelog - **Source:** https://github.com/elevenlabs/skills/blob/main/.agents/skills/update-skills-from-changelog/SKILL.md - **Raw:** https://raw.githubusercontent.com/elevenlabs/skills/main/.agents/skills/update-skills-from-changelog/SKILL.md - **Markdown:** https://skillsdocs.com/elevenlabs/skills/update-skills-from-changelog.md - **Origin:** Credited — installed into this repository, not published from it. - **Licence:** MIT — https://spdx.org/licenses/MIT.html --- name: update-skills-from-changelog description: > Update ElevenLabs agent skills from a merged weekly changelog in elevenlabs-dx, then open a pull request in elevenlabs/skills. Trigger after a changelog merges to main on elevenlabs-dx, or when asked to update skills from changelog YYYY-MM-DD. --- You update ElevenLabs agent skills based on a merged weekly changelog in [elevenlabs-dx](https://github.com/elevenlabs/elevenlabs-dx). Do not read GitHub issues. Open a pull request in this repository when skill files change. Skill files are evergreen source-of-truth documentation for current behavior. Use the changelog to discover what changed, but write final `SKILL.md` and `references/*.md` content as timeless present-tense documentation. Skill files are high-level, task-oriented guidance for working with ElevenLabs. They are not meant to mirror every nuance from the changelog or API reference. Prefer documenting primary workflows, core capabilities, and important configuration surfaces. Usually omit edge cases, precedence chains, persistence mechanics, fallback order, implementation details, and narrow exceptions unless leaving them out would make the skill materially misleading or unusable. ## Workflow 1. Resolve `CHANGELOG_DATE` from the automation trigger or user message. 2. Fetch the merged changelog from `elevenlabs-dx` `main`. 3. Apply the relevance filter. Stop if no skills are affected. 4. Read every affected skill file and reference file. 5. Verify each candidate update against canonical source documentation. 6. Apply only targeted, evergreen skill edits that pass the fit gate. 7. Self-check all edits. 8. Create `skills-update/YYYY-MM-DD`, commit, push, and open a PR. Do not modify skills not implicated by the changelog. ## Step 1: Fetch changelog Determine `CHANGELOG_DATE` (`YYYY-MM-DD`) from the automation trigger or user message. Fetch the merged changelog file: ```bash gh api "repos/elevenlabs/elevenlabs-dx/contents/fern/docs/pages/changelog/${CHANGELOG_DATE}.md?ref=main" \ --jq '.content' | base64 -d > "/tmp/changelog-${CHANGELOG_DATE}.md" ``` If the file does not exist, stop and report that no merged changelog was found for that date. Also read the published page when helpful: `https://elevenlabs.io/docs/changelog#${CHANGELOG_DATE}T00:00:00.000Z` ## Step 2: Apply relevance filter Read `/tmp/changelog-${CHANGELOG_DATE}.md` and map changes against these skills: | Skill | What triggers an update | | --- | --- | | `text-to-speech` | New/deprecated models, new TTS parameters, voice settings changes, output format changes, SDK method signature changes for `text_to_speech.convert()` | | `speech-to-text` | New transcription models, new parameters, changed response schemas, SDK method changes | | `agents` | New LLM providers/models, new tool types, new agent config fields, conversation config schema changes, new CLI commands, widget changes, procedure endpoint/field/type changes, structured procedure validation rules, compile or publish behavior, SDK method changes under `conversational_ai.agents.procedures` or `conversationalAi.agents.procedures` | | `sound-effects` | New generation parameters, model changes, SDK method changes | | `music` | New endpoints, new parameters, model changes | | `voice-isolator` | New parameters, model changes, SDK method changes for `audio_isolation.convert()` | | `speech-engine` | Speech Engine WebSocket API changes, conversation token changes, SDK method changes for real-time voice conversations | | `voice-changer` | New speech-to-speech parameters, model changes, SDK method changes for `speech_to_speech.convert()` or `speechToSpeech.convert()` | | `setup-api-key` | Authentication flow changes, API key dashboard changes, environment variable guidance | A change is relevant if it affects model tables, code examples, parameter documentation, configuration tables, or CLI commands documented in skills. A change is not relevant if it only affects internal/admin APIs, optional fields with no usage-level impact, backward-compatible renames, or pricing/dashboard UI unrelated to the API key setup flow. If no skills are affected, stop successfully without opening a pull request. Report `No skills-relevant changes for CHANGELOG_DATE`. For each relevant item, note the affected skill and affected area, such as model table, code examples, LLM provider table, tools section, CLI section, parameter documentation, or configuration table. ## Step 3: Read current skill files For each affected skill, read: - `{skill}/SKILL.md` - All files in `{skill}/references/` Skill directories: - `text-to-speech/` (`SKILL.md` plus `references/installation.md`, `references/streaming.md`, `references/voice-settings.md`) - `speech-to-text/` (`SKILL.md` plus `references/installation.md`, `references/transcription-options.md`, `references/realtime-server-side.md`, `references/realtime-client-side.md`, `references/realtime-commit-strategies.md`, `references/realtime-events.md`) - `agents/` (`SKILL.md` plus `references/installation.md`, `references/agent-configuration.md`, `references/client-tools.md`, `references/widget-embedding.md`, `references/outbound-calls.md`, `references/using-procedure-api.md`, `references/writing-procedures.md`) - `sound-effects/` (`SKILL.md` plus `references/installation.md`) - `music/` (`SKILL.md` plus `references/installation.md`, `references/api_reference.md`) - `voice-isolator/` (`SKILL.md` plus `references/installation.md`) - `speech-engine/` (`SKILL.md` plus `references/installation.md`, `references/javascript-sdk-reference.md`, `references/python-sdk-reference.md`) - `voice-changer/` (`SKILL.md` plus `references/installation.md`) - `setup-api-key/` (`SKILL.md` only) ## Step 4: Verify source documentation Before editing, fetch and read the actual source material. The changelog tells you what changed; API/reference docs tell you what to document as current behavior. For common areas, start with: - Agents: `https://elevenlabs.io/docs/api-reference/agents/create`, `https://elevenlabs.io/docs/api-reference/agents/update` - TTS: `https://elevenlabs.io/docs/api-reference/text-to-speech/convert` - STT: `https://elevenlabs.io/docs/api-reference/speech-to-text/convert` For every documented field, parameter, schema, enum, endpoint, model ID, or SDK method: - Verify the exact field name, type, nesting, allowed values, and method signature in source documentation. - Never infer schemas from changelog wording. - If a feature appears in the changelog but source docs do not provide enough schema detail, do not write field tables or code examples for it. Put it under `Needs Manual Authoring` in the report. ## Step 5: Decide whether each item belongs Run this fit gate before editing each changelog item: 1. Map to a natural home in an existing section, table, list, or example. 2. Include only primary capabilities, common workflows, or important top-level configuration concepts. 3. Skip secondary nuances: edge cases, precedence rules, persistence details, fallback order, implementation details, narrow exceptions, or deprecation notes. 4. Prefer no-op over forced structure. If no natural home exists, leave skill files unchanged and report it under `No Skill Change Needed`. 5. Add a new section only when the concept is substantial, reusable, user-facing, high-level, and clearly missing from current structure. 6. Prefer the current path. If a field, endpoint, model, package, or pattern replaces another, document the current supported way and keep deprecated context in the report. Good fits: - Add a new supported model row to an existing model table. - Add a new top-level parameter to an existing parameter table. - Update existing Python, JavaScript, and CLI examples when method signatures change. Bad fits: - Insert a standalone sentence between unrelated sections just to mention a changelog item. - Add deprecated fields, removed enum values, old package names, or migration warnings unless the skill already has an explicit migration/troubleshooting section and the change is needed there. - Document internal precedence, local persistence behavior, fallback chains, or rare exception behavior. ## Step 6: Make targeted edits Apply the smallest useful change to the correct file and section. Match existing heading levels, table formats, code block languages, indentation, and naming style. Update patterns: - **Model table:** add, remove, or modify rows in the relevant `SKILL.md` model table. Verify model IDs and descriptions. - **Code examples:** update method signatures, imports, and significant parameters. Keep Python, JavaScript, and CLI examples consistent when all exist. - **LLM provider table:** update `agents/SKILL.md` or `agents/references/agent-configuration.md`. - **Tools section:** update `agents/SKILL.md` with new tool types in the existing style. - **CLI section:** update existing CLI examples in `agents/` files. - **Parameter documentation:** add verified parameters to the relevant parameter list or table. - **Configuration tables:** update field tables in reference files such as `agent-configuration.md` or `voice-settings.md`. - **Output format table:** update the output format table in `text-to-speech/SKILL.md`. Hard rules: - Never invent field names, types, schemas, model IDs, config names, endpoint paths, or example values. - Never write code examples for new features without verifying the exact API shape. - Treat the changelog as discovery input, not skill-file prose. - Skill files must be evergreen. Never mention changelog, issue, PR, release date, "added in", "introduced in", "as of", or "now supports" inside `SKILL.md` or `references/*.md`. - Document current positive workflows, not negative history. - Do not create a new section solely because a changelog bullet exists. - Do not insert orphan content. - Keep tables focused on supported current fields. - If an SDK version bump has no method signature change, update version-specific comments only if such comments already exist. ## Step 7: Self-check before committing Review every change and verify: 1. Every edited field name appears in the source docs read in Step 4. 2. Every code example uses verified parameter names and nesting. 3. No content was inferred from changelog wording alone. 4. No fabricated values remain. 5. No edited skill file references a changelog, issue, PR, release date, or release-history phrasing. 6. Every new heading or section is justified by Step 5. 7. No orphan sentences or forced one-off sections remain. 8. No edited content documents deprecated, removed, or replaced fields solely as negative guidance. 9. Every relevant changelog item is accounted for as one of: docs update, justified new section, `No Skill Change Needed`, or `Needs Manual Authoring`. If any change fails this check, revert that edit and move the item to `Needs Manual Authoring` or `No Skill Change Needed`. ## Step 8: Branch, commit, and pull request Use branch name `skills-update/YYYY-MM-DD`. Before creating a branch, check for an existing open PR and stop if one exists: ```bash gh pr list --repo elevenlabs/skills --head "skills-update/${CHANGELOG_DATE}" --state open --json number --jq 'length == 0' ``` Create the branch from current `origin/main`, commit only if files changed, and push: ```bash git fetch origin main git checkout -b "skills-update/${CHANGELOG_DATE}" origin/main git add -A git diff --cached --quiet || git commit -m "Update skills from changelog ${CHANGELOG_DATE}" git push -u origin "skills-update/${CHANGELOG_DATE}" ``` Write the report to `/tmp/skills-update-report.md`, then open the PR: ```bash gh pr create --repo elevenlabs/skills \ --base main \ --head "skills-update/${CHANGELOG_DATE}" \ --title "Update skills from changelog ${CHANGELOG_DATE}" \ --body-file /tmp/skills-update-report.md ``` If there are no file changes after analysis, do not push or open a PR. Report `No skill file changes needed for CHANGELOG_DATE` with relevant items under `No Skill Change Needed`. ## Report and PR body requirements Write this report and use it as the PR body: ```markdown # Skills Update Report ## Outcome - Changelog: `YYYY-MM-DD` - Branch: `skills-update/YYYY-MM-DD` - Commit: `` - Pull request: `` - Result: `` ## Summary Updates skills based on the merged weekly changelog. If the changelog describes breaking API or SDK changes, add a short warning here describing which examples or docs may need migration guidance. ### Changes - **skill-name**: Brief description of what changed. ### Verification - `field_or_area` in `file.md` - verified against [API reference page](url) ### Needs Manual Authoring List changelog items that were not applied because the schema could not be verified. Include what the changelog said, why it could not be verified, and the source link to check later. If no items apply, write "None." ### No Skill Change Needed List verified changelog items intentionally not added to skill files because they have no natural home or are too low-level for skills. Include what changed, why no edit was appropriate, and the source link. If no items apply, write "None." ### Open Questions List blockers, ambiguous source material, or follow-up items. If no items apply, write "None." ### Source [Changelog YYYY-MM-DD](https://elevenlabs.io/docs/changelog#YYYY-MM-DDT00:00:00.000Z) [Changelog file on GitHub](https://github.com/elevenlabs/elevenlabs-dx/blob/main/fern/docs/pages/changelog/YYYY-MM-DD.md) ``` When run in an automated/headless agent environment with repository write access, opening the pull request is required unless no skill files changed. Return only the completed markdown report in the final response unless the invoking user explicitly asks for extra commentary. ## Important - Do not change YAML frontmatter in skill files unless the changelog specifically requires it. - Keep changelog dates and release-history wording in the report/PR context only. - Prefer no-op over forced structure when a changelog item has no natural home. - A wrong code example is worse than a missing one. - If changelog coverage only gives feature names and high-level descriptions without source docs or schema details, put those items under `Needs Manual Authoring`.