Setting the file. One moment.
Subchapter 1.2
references/client-tools.mdMarkdown20 KBView on GitHub
Extend your agent with custom capabilities. Tools let the agent take actions beyond just talking.
| Type | Execution | Use Case |
|---|---|---|
| Webhook |
| Server-side via HTTP |
| Database queries, API calls, secure operations |
| Client | Browser-side JavaScript | UI updates, local storage, navigation |
| System | Built-in ElevenLabs | End call, transfer, standard actions |
Tools are defined inside conversation_config.agent.prompt. Webhook and client tools go in the tools array. System tools go in built_in_tools:
conversation_config={
"agent": {
"prompt": {
"prompt": "You are helpful.",
"llm": "gemini-2.0-flash",
"tools": [...], # Webhook and client tools
"built_in_tools": {...} # System tools (end_call, transfer, etc.)
}
}
}Execute server-side logic when the agent needs external data or actions.
agent = client.conversational_ai.agents.create(
name="Weather Assistant",
conversation_config={
"agent": {
"prompt": {
"prompt": "You are a helpful assistant that can check the weather.",
"llm": "gemini-2.0-flash",
"tools": [{
"type": "webhook",
"name": "get_weather",
"description": "Get current weather for a city. Use when user asks about weather.",
"api_schema": {
"url": "https://api.example.com/weather",
"method": "POST",
"request_headers": {
"Authorization": "Bearer {{API_KEY}}"
},
"request_body_schema": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g., 'San Francisco'"
},
"units": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature units"
}
},
"required": ["city"]
}
}
}]
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
}
)When the agent calls a webhook tool, ElevenLabs sends:
{
"tool_call_id": "call_abc123",
"tool_name": "get_weather",
"parameters": {
"city": "San Francisco",
"units": "fahrenheit"
},
"conversation_id": "conv_xyz789"
}Your server should respond with:
{
"result": "The weather in San Francisco is 68°F and sunny."
}Or for structured data:
{
"result": {
"temperature": 68,
"condition": "sunny",
"humidity": 45
}
}# Inside conversation_config.agent.prompt.tools:
{
"type": "webhook",
"name": "lookup_order",
"description": "Look up order status by order ID",
"response_timeout_secs": 10,
"api_schema": {
"url": "https://api.mystore.com/orders/lookup",
"method": "POST",
"request_headers": {
"Authorization": "Bearer {{ORDER_API_KEY}}",
"X-Store-ID": "store_123"
},
"request_body_schema": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "Order ID (e.g., ORD-12345)"
}
},
"required": ["order_id"]
}
}
}Use workspace environment variables to keep a single server tool configuration working across
staging and production. {{system_env__label}} works in server tool URLs, secret environment
variables can populate request_headers, and auth-connection environment variables can populate
api_schema.auth_connection. The same environment-variable resolution model also applies to MCP
server connections.
{
"api_schema": {
"url": "https://{{system_env__api_host}}.example.com/orders",
"method": "GET",
"request_headers": {
"X-Api-Key": { "env_var_label": "orders_api_key" }
},
"auth_connection": { "env_var_label": "orders_oauth" }
}
}Workspace auth connections support OAuth2 client credentials, OAuth2 JWT, private key JWT,
basic auth, bearer auth, custom header auth, and mutual TLS (mtls).
System dynamic variables are also available in tool parameters and headers. Use
{{system__conversation_history}} when a webhook or sub-agent needs the full conversation
context as a lazily evaluated JSON history object with user, agent, and tool entries.
| Field | Type | Default | Description |
|---|---|---|---|
response_timeout_secs | int | 20 | Timeout in seconds (5-120) |
interruption_mode | string | "allow" | Controls whether the user can interrupt around this tool call: allow, disable_during_tool, or disable_during_tool_and_turn |
execution_mode | string | "immediate" | immediate, post_tool_speech, or async |
tool_call_sound | string | - | Sound during execution: typing, elevator1-elevator4 |
pre_tool_speech | string | "auto" | Controls whether the agent speaks before execution: auto, force, or off |
tool_error_handling_mode | string | "auto" | auto, summarized, passthrough, or hide |
api_schema.response_filter | object | - | Filters JSON webhook responses before the LLM sees them. Use mode: "allow" with filters dot-paths to keep selected fields, or mode: "hide_all" to hide the response |
MCP server configuration supports the same pre_tool_speech, interruption_mode, execution_mode, and
response_timeout_secs controls at the server level, with per-tool overrides in
tool_config_overrides. Set a per-tool tool_call_sound override to "off" to silence that tool
while retaining the server default for other tools. MCP timeouts default to 30 seconds and must be
5-300 seconds.
Set request_meta on an MCP server configuration to send entries in the MCP _meta field of
tools/call requests. Values can be JSON scalars or references to workspace secrets, dynamic
variables, or environment variables that resolve for each call.
Note: The default api_schema.method is GET. Always set "method": "POST" explicitly for webhook tools that send request bodies.
app.post("/webhook/get_weather", async (req, res) => {
const { parameters, conversation_id } = req.body;
const { city, units = "fahrenheit" } = parameters;
// Fetch weather from your data source
const weather = await weatherService.get(city, units);
res.json({
result: `It's ${weather.temp}°${units === "celsius" ? "C" : "F"} and ${weather.condition} in ${city}.`,
});
});@app.post("/webhook/get_weather")
async def get_weather(request: Request):
data = await request.json()
city = data["parameters"]["city"]
units = data["parameters"].get("units", "fahrenheit")
# Fetch weather from your data source
weather = weather_service.get(city, units)
return {
"result": f"It's {weather['temp']}°{'C' if units == 'celsius' else 'F'} and {weather['condition']} in {city}."
}Execute JavaScript in the user’s browser. Useful for UI updates, navigation, or accessing browser APIs.
Client tools are registered when starting a conversation:
import { Conversation } from "@elevenlabs/client";
const conversation = await Conversation.startSession({
agentId: "your-agent-id",
clientTools: {
show_product: async ({ productId }) => {
// Update UI to show product
const modal = document.getElementById("product-modal");
modal.innerHTML = await fetchProductCard(productId);
modal.showModal();
return { success: true, message: "Showing product" };
},
navigate_to: async ({ page }) => {
// Navigate to a page
window.location.href = `/${page}`;
return { success: true };
},
save_preference: async ({ key, value }) => {
// Store in localStorage
localStorage.setItem(key, value);
return { saved: true };
},
},
});When you use the React SDK, wrap your component tree in ConversationProvider and register
client tools from components with useConversationClientTool. Handlers are cleaned up
automatically on unmount and always use the latest closure value. Prefer granular hooks such as
useConversationControls and useConversationStatus for the session UI; useConversation
remains available when you want the full conversation object in one hook.
import {
ConversationProvider,
useConversationClientTool,
useConversationControls,
useConversationStatus,
} from "@elevenlabs/react";
function Storefront() {
useConversationClientTool("show_product", async ({ productId }) => {
const modal = document.getElementById("product-modal");
modal.innerHTML = await fetchProductCard(productId);
modal.showModal();
return { success: true };
});
const { startSession, endSession } = useConversationControls();
const { status } = useConversationStatus();
if (status === "connected") {
return <button onClick={endSession}>End</button>;
}
return (
<button onClick={() => startSession({ agentId: "your-agent-id" })}>
Start
</button>
);
}
function App() {
return (
<ConversationProvider>
<Storefront />
</ConversationProvider>
);
}Tell the agent about available client tools in conversation_config.agent.prompt.tools:
agent = client.conversational_ai.agents.create(
name="Shopping Assistant",
conversation_config={
"agent": {
"prompt": {
"prompt": """You are a shopping assistant.
When users want to see a product, use show_product.
When users want to go somewhere, use navigate_to.""",
"llm": "gemini-2.0-flash",
"tools": [
{
"type": "client",
"name": "show_product",
"description": "Display a product card to the user",
"parameters": {
"type": "object",
"properties": {
"productId": {
"type": "string",
"description": "Product ID to display"
}
},
"required": ["productId"]
}
},
{
"type": "client",
"name": "navigate_to",
"description": "Navigate user to a different page",
"parameters": {
"type": "object",
"properties": {
"page": {
"type": "string",
"enum": ["cart", "checkout", "account", "home"],
"description": "Page to navigate to"
}
},
"required": ["page"]
}
}
]
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
}
)| Field | Type | Default | Description |
|---|---|---|---|
expects_response | bool | false | Whether the tool returns data to the agent |
Return data that the agent can use in conversation:
clientTools: {
check_cart: async () => {
const cart = JSON.parse(localStorage.getItem("cart") || "[]");
return {
itemCount: cart.length,
total: cart.reduce((sum, item) => sum + item.price, 0),
items: cart.map((item) => item.name),
};
};
}The agent receives this data and can say: “You have 3 items in your cart totaling $45.99.”
Built-in tools provided by ElevenLabs. These are configured in conversation_config.agent.prompt.built_in_tools (not in the tools array):
"built_in_tools": {
"end_call": {},
"transfer_to_number": {...},
"transfer_to_agent": {...},
"language_detection": {},
"skip_turn": {},
"voicemail_detection": {...},
"play_keypad_touch_tone": {}
}Current API schemas also expose agent_prompt_change, memory_entry_create, memory_entry_delete, memory_entry_search, and memory_entry_update in built_in_tools.
Set only_at_conversation_start: true on the language_detection tool to constrain language
switching when no switch occurs in the first two user turns. Leave it false when the conversation
must remain able to switch languages later.
Ends the current conversation:
"built_in_tools": {
"end_call": {}
}The agent can say “Goodbye!” and then end the call programmatically.
Transfer to a phone number (requires telephony integration):
"built_in_tools": {
"transfer_to_number": {
"transfers": [{
"transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
"condition": "User asks to speak with a human agent"
}]
}
}Transfer to another ElevenLabs agent or workflow node:
"built_in_tools": {
"transfer_to_agent": {
"transfers": [{
"agent_id": "other-agent-id",
"node_id": "destination-workflow-node-id",
"preserve_client_tts_overrides": true,
"condition": "User asks about sales"
}]
}
}Use node_id when the transfer should start at a specific workflow node. Omit
agent_id when the transfer stays within the current agent’s workflow.
Set preserve_client_tts_overrides when client-side TTS overrides should continue
after the transfer.
Write clear descriptions so the LLM knows when to use tools:
# Good - specific and actionable
"description": "Look up order status. Use when customer asks about their order, delivery, or shipping."
# Bad - vague
"description": "Order tool"Help the LLM extract correct values:
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "Order ID in format ORD-XXXXX (e.g., ORD-12345)"
},
"email": {
"type": "string",
"description": "Customer email address for verification"
}
}
}For optional tool parameters that should never be sent in the request payload, set
is_omitted: true on the JSON schema property. Do not combine it with description,
dynamic_variable, is_system_provided, or constant_value.
For an LLM-supplied parameter that must match a runtime list, set
allowed_values: {"dynamic_variable": "allowed_ids"}. The dynamic variable must resolve to a
JSON array. Use allowed_values only with a description-sourced property; do not combine it with
dynamic_variable, is_system_provided, constant_value, or is_omitted.
Configure how tool errors are shared with the agent using tool_error_handling_mode:
| Mode | Behavior |
|---|---|
auto | ElevenLabs automatically decides how to handle errors |
summarized | Errors are summarized before being sent to the agent |
passthrough | Full error details are passed to the agent |
hide | Errors are hidden from the agent |
Return helpful error messages:
// Server webhook
app.post("/webhook/lookup_order", async (req, res) => {
const { order_id } = req.body.parameters;
const order = await db.orders.find(order_id);
if (!order) {
return res.json({
result: {
error: true,
message: `Order ${order_id} not found. Please verify the order ID.`,
},
});
}
res.json({ result: order });
});Set reasonable timeouts for webhooks using response_timeout_secs (5-120 seconds, default 20). MCP server tool calls use the same field with a 30-second default and a 5-300 second range:
{
"type": "webhook",
"name": "slow_operation",
"description": "Run a slow operation",
"response_timeout_secs": 30,
"api_schema": {
"url": "https://api.example.com/slow-operation",
"method": "POST"
}
}agent = client.conversational_ai.agents.create(
name="E-commerce Assistant",
conversation_config={
"agent": {
"first_message": "Hi! How can I help you today?",
"language": "en",
"prompt": {
"prompt": """You are an e-commerce support assistant.
Available actions:
- lookup_order: Check order status
- show_product: Display products to customer
- end_call: End conversation politely
- transfer_to_number: Transfer to human support
Always verify order ID before lookup. Offer transfer for complex issues.""",
"llm": "gemini-2.0-flash",
"tools": [
# Webhook: Server-side order lookup
{
"type": "webhook",
"name": "lookup_order",
"description": "Look up order status by order ID or email",
"api_schema": {
"url": "https://api.mystore.com/orders/lookup",
"method": "POST",
"request_headers": {"Authorization": "Bearer {{API_KEY}}"},
"request_body_schema": {
"type": "object",
"properties": {
"order_id": {"type": "string"},
"email": {"type": "string"}
}
}
}
},
# Client: Browser-side product display
{
"type": "client",
"name": "show_product",
"description": "Display product details to the customer",
"parameters": {
"type": "object",
"properties": {
"product_id": {"type": "string"}
},
"required": ["product_id"]
}
}
],
"built_in_tools": {
"end_call": {},
"transfer_to_number": {
"transfers": [{
"transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
"condition": "User asks for human support"
}]
}
}
}
},
"tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_flash_v2_5"}
}
)