Setting the file. One moment.
Skill 35 · N8n:create Agent Builder Eval
Subchapter 35.1
local-setup.mdMarkdown3 KBView on GitHub
Generate a key on the
LangTracer API keys page (opens in a new tab).
Add it to .env.local at the repository root:
LANGTRACER_URL=https://lang-tracer.n8n-maintenance.workers.dev
LANGTRACER_API_KEY=<generated-key>Do not paste the key into chat or put it in a command. The main skill checks that both variables exist before it creates an eval.
Create .env.local at the repository root. Use one Instance AI model provider
and one sandbox provider.
N8N_INSTANCE_AI_MODEL=anthropic/claude-sonnet-4-6
N8N_INSTANCE_AI_MODEL_API_KEY=<model-key>
ANTHROPIC_API_KEY=<same-model-key>
N8N_AI_ASSISTANT_BASE_URL=
N8N_INSTANCE_AI_SANDBOX_ENABLED=true
N8N_INSTANCE_AI_SANDBOX_PROVIDER=daytona
DAYTONA_API_URL=https://app.daytona.io/api
DAYTONA_API_KEY=<daytona-key>
N8N_EVAL_EMAIL=nathan@n8n.io
N8N_EVAL_PASSWORD=PlaywrightTest123Use the n8n sandbox service instead of Daytona when it is available. Set
N8N_INSTANCE_AI_SANDBOX_PROVIDER=n8n-sandbox,
N8N_SANDBOX_SERVICE_URL, and, when required,
N8N_SANDBOX_SERVICE_API_KEY.
LangSmith is optional. Unset LANGSMITH_API_KEY and LANGSMITH_TRACING for a
throwaway direct run. This keeps the run out of the shared LangSmith dataset.
From the repository root:
(
pnpm --filter n8n... build > build.log 2>&1
build_status=$?
tail -n 20 build.log
exit "$build_status"
)Restart the instance after each build. A running process keeps the old code in memory.
Create a temporary user directory:
mktemp -d /tmp/n8n-agent-eval.XXXXXXUse the printed path as N8N_USER_FOLDER. Start the built CLI from
packages/cli:
N8N_PORT=5680 \
N8N_RUNNERS_BROKER_PORT=5681 \
N8N_USER_FOLDER=<printed-temp-directory> \
E2E_TESTS=true \
N8N_ENABLED_MODULES=instance-ai,agents \
N8N_AI_ENABLED=true \
pnpm exec dotenvx run -f ../../.env.local -- pnpm startUse different ports when 5680 or 5681 is busy.
Run this once after the isolated instance starts:
curl -sf -X POST http://localhost:5680/rest/e2e/reset \
-H 'Content-Type: application/json' \
-d '{"owner":{"email":"nathan@n8n.io","password":"PlaywrightTest123","firstName":"Eval","lastName":"Owner"},"admin":{"email":"admin@n8n.io","password":"PlaywrightTest123","firstName":"Admin","lastName":"User"},"members":[],"chat":{"email":"chat@n8n.io","password":"PlaywrightTest123","firstName":"Chat","lastName":"User"}}'From packages/@n8n/instance-ai:
pnpm exec dotenvx run -f ../../../.env.local -- \
env -u LANGSMITH_API_KEY -u LANGSMITH_TRACING \
pnpm eval:instance-ai \
--base-url http://localhost:5680 \
--filter <slug> \
--tier agents \
--concurrency 1 \
--verboseThe run writes eval-results.json, an HTML report, and a PR-comment preview.
The warnings about disabled run-debug capture are benign.