🤖 AI Engineering · Fine-tuning & Adaptation

Prepare supervised fine-tuning datasets

Clean, deduplicate, and balance instruction-response pairs so fine-tuning learns behavior rather than noise.

advanced~60 minAI EngineersML EngineersLLM App Developers

Steps

1Define the target behavior as 5 crisp capability statements before collecting data
2Deduplicate near-identical pairs; duplicates amplify artifacts
3Balance coverage across capabilities, lengths, and difficulty tiers
4Hold out 3% as an untouched eval split mirroring real traffic
5Scrub PII and secrets with automated scans plus human spot checks
6Start LoRA-scale: 500-2000 high-quality pairs beat 50k scraped ones

Common Pitfalls

▲Training on outputs of the same model you are tuning, compounding errors
▲Response style inconsistent across contributors

Success Signals

✓Eval-split win-rate vs base model above 70%
✓Zero PII findings in pre-flight scan

Commands

Install with skills CLI

$ npx skills add aniruddhaadak80/skills --skill fine-tuning-adaptation-dataset-prep-sft

Install globally

$ npx skills add aniruddhaadak80/skills --skill fine-tuning-adaptation-dataset-prep-sft -g

Tags

#fine-tuning#datasets#training#ai-engineering#fine-tuning-adaptation

Install this skill

$ npx skills add aniruddhaadak80/skills --skill fine-tuning-adaptation-dataset-prep-sft -a opencode

SKILL.md

# Prepare supervised fine-tuning datasets

> Clean, deduplicate, and balance instruction-response pairs so fine-tuning learns behavior rather than noise.

## When to Use This Skill

Clean, deduplicate, and balance instruction-response pairs so fine-tuning learns behavior rather than noise.

## Steps

1. Define the target behavior as 5 crisp capability statements before collecting data
2. Deduplicate near-identical pairs; duplicates amplify artifacts
3. Balance coverage across capabilities, lengths, and difficulty tiers
4. Hold out 3% as an untouched eval split mirroring real traffic
5. Scrub PII and secrets with automated scans plus human spot checks
6. Start LoRA-scale: 500-2000 high-quality pairs beat 50k scraped ones

## Common Pitfalls

- Training on outputs of the same model you are tuning, compounding errors
- Response style inconsistent across contributors

View source on GitHub ↗

Pick the cheapest adaptation layer that solves the problem using a decision ladder.

🤖 AI Engineering·~20m

Open →