Setting the file. One moment.
Chapter 01 · MongoDB Atlas Stream Processing
Subchapter 1.6
references/sizing-and-parallelism.mdMarkdown6 KBView on GitHub
| Tier | vCPU | RAM | Bandwidth | Max Parallelism | Kafka Partitions | Use case |
|---|---|---|---|---|---|---|
| SP2 | 0.25 | 512MB | 50 Mbps | 1 | 32 | Minimal filtering, testing |
| SP5 | 0.5 | 1GB | 125 Mbps | 2 | 64 | Simple filtering and routing |
| SP10 | 1 | 2GB | 200 Mbps | 8 | Unlimited | Moderate workloads, joins, grouping |
| SP30 | 2 | 8GB | 750 Mbps | 16 | Unlimited | Windows, JavaScript UDFs, production |
| SP50 | 8 | 32GB | 2500 Mbps | 64 | Unlimited | High throughput, large window state |
Memory rule: 20% is reserved for overhead. User state (window accumulation, sort buffers) must stay below 80% of tier RAM. Exceeding this causes OOM failure.
Every stage in a pipeline runs with default parallelism: 1. This base level is included in your tier at no additional cost.
When you need higher throughput for specific stages, increase their parallelism beyond 1. Only values > 1 count toward your tier’s maximum.
Stages that commonly benefit from parallelism:
$merge — concurrent writes to Atlas$lookup — concurrent reads for enrichment$https — concurrent API callsFormula: Total Parallelism = sum of (parallelism - 1) for all stages where parallelism > 1
If Total Parallelism = 0: → SP2 (max 1)
If Total Parallelism = 1: → SP5 (max 2)
If Total Parallelism ≤ 8: → SP10 (max 8)
If Total Parallelism ≤ 16: → SP30 (max 16)
If Total Parallelism ≤ 64: → SP50 (max 64)Simple pipeline (all parallelism = 1):
$source: parallelism = 1 (does not count)
$match: parallelism = 1 (does not count)
$merge: parallelism = 1 (does not count)
Total = 0 → SP2Medium pipeline:
$source: parallelism = 1 (does not count)
$match: parallelism = 1 (does not count)
$lookup: parallelism = 4 (counts as 3)
$merge: parallelism = 4 (counts as 3)
Total = 3 + 3 = 6 → SP10 (max 8)Complex pipeline:
$source: parallelism = 1 (does not count)
$https: parallelism = 6 (counts as 5)
$merge: parallelism = 8 (counts as 7)
Total = 5 + 7 = 12 → SP30 (max 16)If you specify a tier too small for the pipeline’s parallelism, the API returns:
"Operator parallelism requested exceeds limit for this tier.
(Requested: X, Limit: Y). Minimum tier for this workload: SPxx or larger."Solution: Use atlas-streams-manage → stop-processor, then start-processor with a higher tier value.
When parallelism is all default (1), choose tier based on pipeline complexity:
| Pipeline feature | Complexity weight | Minimum tier |
|---|---|---|
Simple $match + $project only | Low | SP2-SP5 |
$addFields with expressions | Low-Medium | SP5-SP10 |
$lookup or $https enrichment | Medium | SP10 |
$group aggregation | Medium | SP10 |
$tumblingWindow or $hoppingWindow | Medium-High | SP10-SP30 |
$sessionWindow | High | SP30 |
$function (JavaScript UDFs) | High | SP30+ |
| Large window state (many unique keys) | Very High | SP30-SP50 |
| Multiple windows or chained enrichment | Very High | SP50 |
For automated tier recommendation, score the pipeline:
| Feature | Points |
|---|---|
$function (JavaScript) | +40 |
Window operations ($tumblingWindow, $hoppingWindow, $sessionWindow) | +30 |
$lookup or $https enrichment | +20 |
$group aggregation | +15 |
| Kafka source integration | +15 |
$sort operations | +10 |
| Pipeline has 5+ stages | +5 |
| Pipeline has 8+ stages | +10 |
| Pipeline has 12+ stages | +20 |
Score → Tier mapping:
Always take the higher of complexity-driven vs parallelism-driven tier recommendations.
Charges are per-hour, calculated per-second, only while the processor is running.
start-processor begins billingstop-processor stops billingWhat’s included in the tier price:
Additional costs (separate from tier):
For current pricing: https://www.mongodb.com/docs/atlas/billing/stream-processing-costs/ (opens in a new tab)
atlas-streams-build → resource: "processor" with autoStart: trueatlas-streams-discover → diagnose-processormemoryUsageBytes:
atlas-streams-manage → stop-processoratlas-streams-manage → start-processor with tier overrideFor workloads with predictable traffic patterns, adjust tiers by time of day:
| Period | Tier | Rationale |
|---|---|---|
| Peak hours (business hours) | SP30-SP50 | Handle full volume |
| Off-peak hours | SP10-SP30 | Reduced volume |
| Maintenance windows | SP2-SP10 | Minimal processing |
To change tiers: stop-processor → start-processor with new tier value. Note: resumeFromCheckpoint: true (default) preserves state across tier changes.