Gotchas

HTTP 200 / send() resolves, but no data in the sink. Causes:

1. Schema validation failure — structured streams accept then silently drop invalid events during processing. Validate client-side (Zod) and monitor pipelinesUserErrorsAdaptiveGroups.
2. First-flush warm-up — first data takes 3–7 minutes (warm-up + namespace/table creation) even with --roll-interval 10. Poll ≥5 min in tests.
3. Roll interval not elapsed — default 300s.
4. Silent sink failure — deleted bucket or expired token. Check recordsWritten > 0 but filesWritten = 0; inspect failure_reason via GET /pipelines/{id}.

Everything is immutable

Cannot modify stream schema, pipeline SQL, or sink config — delete and recreate. Use version naming (events_v1) and keep SQL in version control.

Worker binding undefined (env.MY_STREAM)

1. Use the stream ID, not pipeline ID, in wrangler.jsonc.
2. Binding field is "stream" (June 2026); old "pipeline" still works.
3. Redeploy after adding the binding.

REST API field names ≠ CLI flags

r2_data_catalog vs --type basin-catalog, table_name vs --table, token vs --catalog-token, and format is required in REST but implied in CLI. See configuration.md.

wrangler basin pipelines delete defaults to “no”

Non-interactive environments answer “no” automatically — use REST DELETE for CI/automation.

Behavioral Notes

• __ingest_ts auto-added (TIMESTAMP, day-partitioned). Don’t put it in your schema.
• Sinks can’t target existing tables — the sink creates its own. Use PySpark to write to existing tables.
• JSON-only input — no Avro/Protobuf/CSV.
• Naming: streams/sinks/pipelines use underscores; buckets use hyphens.
• Metrics lag 5–10 min after creation.
• Pipeline SQL is row-level only — no GROUP BY/aggregation/window functions (do aggregation in Basin SQL (opens in a new tab) at query time). CTEs and UNNEST are supported.

Debug Checklist

• Not done: Stream exists: wrangler basin pipelines streams list
• Not done: Pipeline running (not initializing/failed): GET /pipelines/{id}, check failure_reason
• Not done: SQL matches schema; sink token valid; bucket + catalog exist
• Not done: Worker redeployed; binding uses stream ID under "stream"
• Not done: Waited ≥5 min (first flush)
• Not done: Sink metrics: filesWritten > 0; error metrics show no drops

See Also

• configuration.md · api.md · patterns.md