Setting the file. One moment.
Subchapter 31.6
references/hyperparameters.mdMarkdown3 KBView on GitHub
Scripts
scripts/11 files| Parameter | What it controls | Default | Typical range |
|---|---|---|---|
| Epochs | Passes through data | 2 | 1–5 |
| Learning rate multiplier | Weight change aggressiveness | 1.0 | 0.1–2.0 |
| Batch size | Examples per gradient step | Model-dependent | 4–32 |
| Dataset size | Recommended epochs |
|---|---|
| < 100 examples | 3–5 |
| 100–500 examples | 2–3 |
| 500–2,000 examples | 1–2 |
| > 2,000 examples | 1 |
beta (default 0.1): Alignment strength. Lower = more conservative.l2_multiplier (default 0.1): Regularization to prevent drift from base model.| Run | Epochs | LR | Why |
|---|---|---|---|
| 1 | 2 | 1.0 | Baseline |
| 2 | 2 | 0.5 | Conservative |
| 3 | 2 | 1.5 | Aggressive |
| 4 | 3 | 1.0 | More training |
| 5 | 1 | 1.0 | Minimal intervention |
When overfitting (val loss rises after epoch 2): deploy the epoch-2 checkpoint directly instead of retraining. Azure saves checkpoints at each epoch boundary.
checkpoints = client.fine_tuning.jobs.checkpoints.list(job_id)
for cp in checkpoints.data:
print(f"Step {cp.step_number}: val_loss={cp.metrics.valid_loss}")| Model | Recommended Start | Notes |
|---|---|---|
| gpt-4.1-mini | 2ep, lr=0.5–1.0 | Very capable base; small nudges work |
| gpt-4.1-nano | 2–3ep, lr=1.0–1.5 | Smaller capacity, needs more epochs |
| gpt-oss-20b | 2ep, lr=0.2–0.5 | Lower LR critical; deployment may need capacity=100 |
| o4-mini (RFT) | Grader quality > HPs | Focus on grader, not HP sweep |
All OSS models require trainingType: "globalStandard" in the API request.
| Model | Recommended Start | Best Found | Notes |
|---|---|---|---|
| Ministral-3B | 5ep, lr=1.0 | 10ep, lr=0.5 | Small model, slow convergence |
| gpt-oss-20b | 2ep, lr=0.3 | 2ep, lr=0.3 | lr=1.0 overfits quickly |
| Llama-3.3-70B | 3ep, lr=0.3 | 5ep, lr=0.5 | lr=2.0 causes catastrophic degradation |
| Qwen-3-32B | 3ep, lr=0.3 | 3ep, lr=0.3 | Most fragile — more data can hurt |
Key patterns: OSS models need 2–5× more epochs than nano. Lower LR (0.3–0.5) is safer. More data doesn’t always help.
| Parameter | Description | Recommended Start |
|---|---|---|
reasoning_effort | "low", "medium", "high" | "medium" |
compute_multiplier | Scales rollouts per step | 1.5 |
learning_rate_multiplier | Scales LR | 1.0 |
n_epochs | Data passes | 2–3 |
eval_interval | Eval every N steps | 5 |
eval_samples | Validation examples per eval | 10 |
max_episode_steps | Max tool calls + reasoning steps | 5–10 |