Uma Accounting

The Four Terms

Total footprint ≈ weights + optimizer states + gradients + activations, plus a near-zero term for LoRA/QLoRA adapters. Work each term from parameter count and dtype, then sum.

1. Weights

params × bytes/param, by dtype:

A 70B model in bf16 is ~140GB — already over the pool before anything else loads. The same model in 4-bit (QLoRA) is ~35GB, which is why QLoRA, not bf16, is what makes 70B-class models reachable on Spark at all.

2. Optimizer states

Full fine-tuning carries optimizer state for every trainable parameter; LoRA and QLoRA carry it only for the adapter parameters, which is why this term is negligible for them regardless of base model size.

adamw_8bit is the Unsloth default for a reason on a 128GB box — the fp32 variant roughly quadruples this term for any run that isn’t LoRA/QLoRA-adapter-only.

3. Gradients

Same dtype as the compute precision — typically bf16, so 2 bytes/param — and, like optimizer state, only for trainable parameters. Full fine-tuning pays this for every weight; LoRA and QLoRA pay it only for the adapter, since the frozen base weights never accumulate a gradient.

4. Activations

The hardest term to pin to a single number — it scales with batch size, sequence/packing length, and architecture (attention variant, hidden size, layer count), not just parameter count. Two levers matter more than exact estimation:

• Gradient checkpointing trades recompute for memory: expect roughly 30% savings on this term versus no checkpointing, at the cost of a recompute pass per checkpointed segment. Unsloth’s use_gradient_checkpointing="unsloth" is the default for this reason.
• Packing/sequence length is the more direct lever than batch size for this term — see the OOM Ladder in SKILL.md for why packing length is the preferred first cut over batch size.

5. LoRA/QLoRA adapter overhead

A rank-r adapter on a linear layer adds r × (in + out) parameters — A is r×in and B is out×r, so the two matrices together contribute r·in + r·out. At the rank sizes in normal use (1–32 for RL, up to ~256 for SFT-at-scale), this is a small fraction of a percent of base model size — round it to zero in the worksheet unless an unusually high rank is in play.

Anchors

Known-working combinations on a single Spark, to sanity check a new plan against rather than trusting the formula in isolation:

Treat “ceiling” entries as the largest class that fit in practice, not a hard architectural limit — a smaller batch, shorter packing, or a leaner optimizer can sometimes push slightly past one of these, and a heavier configuration of the same model class can fail well under it. Re-derive from the four terms above for anything outside these four reference points, and confirm with the OOM Ladder in SKILL.md if the estimate turns out optimistic.