1{2 "skill_name": "clerk-webhooks",3 "evals": [4 {5 "id": 1,6 "prompt": "every time a user signs up on our app i need to create a row in our users table in postgres (we use prisma). the table has id, email, name, clerkId, createdAt columns. can you set up the webhook handler? we're on next.js app router",7 "expected_output": "Creates webhook route handler at app/api/webhooks/route.ts, verifies webhook signature, handles user.created event, inserts into Prisma users table",8 "files": [],9 "assertions": [
"Creates route handler at app/api/webhooks/route.ts or similar path"
,
11 "Uses POST method handler (not GET)",
12 "Uses verifyWebhook from @clerk/nextjs/webhooks for signature verification",
13 "Handles user.created event type and extracts email, name, id from payload",
14 "Inserts into database using Prisma with the correct column mapping (clerkId = evt.data.id)",
15 "Mentions that webhook route must be excluded from Clerk middleware (public route)"
16 ]
17 },
18 {
19 "id": 2,
20 "prompt": "i want to send a welcome email via resend when someone creates an account. also want to notify our #new-users slack channel. we already have resend and slack sdks installed",
21 "expected_output": "Webhook handler for user.created that sends welcome email via Resend and posts to Slack",
22 "files": [],
23 "assertions": [
24 "Listens for user.created webhook event",
25 "Extracts user email and name from webhook payload",
26 "Calls Resend API to send welcome email",
27 "Posts notification to Slack channel",
28 "Includes webhook signature verification",
29 "Does NOT skip verification even though it's a notification-only handler"
30 ]
31 },
32 {
33 "id": 3,
34 "prompt": "we're getting 401 errors on our webhook endpoint. the clerk dashboard shows the webhooks are being sent but our app keeps rejecting them. we're using express not next.js. heres our current code:\n\napp.post('/webhooks/clerk', express.json(), (req, res) => {\n const evt = req.body;\n console.log(evt);\n res.json({ received: true });\n});",
35 "expected_output": "Identifies missing webhook verification, fixes express body parsing (needs raw body), adds verifyWebhook from @clerk/express/webhooks",
36 "files": [],
37 "assertions": [
38 "Identifies that express.json() is wrong - needs raw body for signature verification",
39 "Shows express.raw({ type: 'application/json' }) instead of express.json() for the webhook route",
40 "Uses verifyWebhook from @clerk/express/webhooks (not raw svix Webhook class)",
41 "Preserves the existing route path /webhooks/clerk",
43 "Returns 200 status on success (not just res.json)"
44 ]
45 },
46 {
47 "id": 4,
48 "prompt": "our app has clerk organizations and we need to sync org membership changes to our database. when someone joins or leaves an org we need to update our team_members table. also when an org is created we need to create a workspace record",
49 "expected_output": "Webhook handler for organizationMembership and organization events with database sync",