Training Types · Finetuning · microsoft/azure-skills · Skills Docs

Training Types · Finetuning · microsoft/azure-skills · Skills Docs

Factor	SFT	DPO	RFT
Best for	Teaching a new skill or format	Aligning preferences/style	Improving reasoning chains
Data needed	Input–output pairs	Chosen/rejected pairs	Prompts + grading function
Data volume	50–5,000 examples	500–5,000 pairs	200–2,000 prompts
Effort to prepare data	Low	High (need contrasting pairs)	Medium (need grader, not outputs)
Risk of regression	Low	Medium	High (sensitive to grader quality)
Typical improvement	5–30% on task metrics	Subtle style/safety shifts	0–15% on reasoning tasks
Supported models	Most models	Select models	o4-mini

├─ Do you have labeled input–output pairs?
│  ├─ Yes → SFT
│  └─ No
│     ├─ Can you write a grading function? → RFT
│     └─ Can you rank "good" vs "bad" outputs? → DPO
│
After SFT:
├─ Results good enough? → Ship it
├─ Need style refinement? → DPO on top of SFT model
└─ Reasoning needs improvement? → RFT (if model supports it)

Model	SFT	DPO	RFT	Vision FT
gpt-4.1	✅	✅	❌	✅
gpt-4.1-mini	✅	❌	❌	❌
gpt-4.1-nano	✅	❌	❌	❌
gpt-4o (2024-08-06)	✅	✅	❌	✅
gpt-4o-mini	✅	❌	❌	❌
o4-mini	❌	❌	✅	❌
gpt-5	❌	❌	✅ ⚠️	❌
gpt-oss-20b	✅	❌	❌	❌
Ministral-3B	✅	❌	❌	❌
Llama-3.3-70B	✅	❌	❌	❌
Qwen-3-32B	✅	❌	❌	❌