Train Dpo Example

1#!/usr/bin/env python3
2# /// script
3# requires-python = ">=3.10"
4# dependencies = [
5#     "trl>=0.12.0",
6#     "transformers>=4.36.0",
7#     "accelerate>=0.24.0",
8#     "trackio",
9# ]
10# ///
11
12"""
13Production-ready DPO training example for preference learning.
14
15DPO (Direct Preference Optimization) trains models on preference pairs
16(chosen vs rejected responses) without requiring a reward model.