Dataset Creation

Write examples by hand, collect from production logs, or adapt existing datasets.

When to use:

• You have real-world examples (production logs, support tickets, labeled data)
• Your task requires domain expertise an LLM can’t reliably generate
• You need a gold-standard evaluation set (always curate manually)

Tips:

• Start with 10-20 examples to establish quality standards and format consistency
• These seed examples also serve as the foundation of your evaluation test set
• For RFT, you only need prompts + expected answers — no model responses needed

Approach 2: LLM Augmentation

Expand a small curated dataset through rephrasing — generating diverse variations while keeping the same expected answer. Especially useful for RFT.

When to use:

• Well-defined task with clear correct answers
• You can write quality examples but need more volume
• Diversity of phrasing matters more than diversity of scenarios

Workflow:

1. Write base examples with correct expected answers
2. For each, use an LLM to generate rephrasings varying tone, detail, and wording
3. Each rephrasing gets the same expected answer — only the phrasing changes
4. Validate the augmented dataset

Rephrasing prompt:

A cheap model (gpt-4.1-mini) works well — no new ground truth needed, just phrasing diversity.

Approach 3: Synthetic Generation

Generate training data from scratch using LLM prompts.

1. Define topic/scenario categories for diversity
2. Generate prompts from an LLM
3. Generate responses (or preferred/non-preferred pairs for DPO)
4. Grade quality with an LLM judge
5. Filter to a quality threshold
6. Split into train/validation/test sets
7. Write JSONL in the correct format (see references/dataset-formats.md)

Quality Checklist

Before training, verify:

• Not done: No duplicates: Exact or near-duplicate examples waste budget
• Not done: Balanced distribution: Topics, difficulty, output lengths well-distributed
• Not done: Consistent formatting: All examples follow the same structure
• Not done: Correct outputs: Spot-check 20 random examples manually
• Not done: Reasonable lengths: No extremely short or extremely long outputs
• Not done: Clean text: No encoding errors, garbled text, or template artifacts

Dataset Size vs. Quality

From experiments:

• 335 high-quality examples (carefully curated) → best combined eval score (9.15)
• 1,576 examples (broader but noisier) → higher correctness but lower conciseness (8.53)

Takeaway: A small, pristine dataset usually beats a large, noisy one. Quality filter aggressively.