🧮 ML Research Engineering · Experimentation Rigor

Reproduce papers without drowning

Staged reproduction from inference-first to full retrain with deviation journals.

advanced~50 minResearch EngineersML ScientistsPhD Researchers

Steps

1Stage 0: run authors' code if available; record environment verbatim
2Stage 1: reimplement inference on their checkpoint; compare outputs numerically
3Stage 2: retrain smallest configuration matching reported metrics ±noise band
4Journal EVERY deviation from paper text with suspected impact
5Contact authors once with specific questions — response rate rewards clarity
6Publish reproduction report regardless of outcome; negatives are valuable

Common Pitfalls

▲Jumping straight to full-scale retraining
▲Undocumented 'small' tweaks compounding into irreproducibility

Commands

Install with skills CLI

$ npx skills add aniruddhaadak80/skills --skill experimentation-rigor-paper-reproduction-protocol

Install globally

$ npx skills add aniruddhaadak80/skills --skill experimentation-rigor-paper-reproduction-protocol -g

Tags

#reproduction#papers#methodology#ml-research#experimentation-rigor

Install this skill

$ npx skills add aniruddhaadak80/skills --skill experimentation-rigor-paper-reproduction-protocol -a opencode

SKILL.md

# Reproduce papers without drowning

> Staged reproduction from inference-first to full retrain with deviation journals.

## When to Use This Skill

Staged reproduction from inference-first to full retrain with deviation journals.

## Steps

1. Stage 0: run authors' code if available; record environment verbatim
2. Stage 1: reimplement inference on their checkpoint; compare outputs numerically
3. Stage 2: retrain smallest configuration matching reported metrics ±noise band
4. Journal EVERY deviation from paper text with suspected impact
5. Contact authors once with specific questions — response rate rewards clarity
6. Publish reproduction report regardless of outcome; negatives are valuable

## Common Pitfalls

- Jumping straight to full-scale retraining
- Undocumented 'small' tweaks compounding into irreproducibility

View source on GitHub ↗

One-factor-at-a-time with matched budgets separating real gains from tuning luck.

🧮 ML Research Engineering·~35m

Open →

Keep experiment tracking trustworthyfoundation

Every run reproducible from logged config + code version + data snapshot reference.

🧮 ML Research Engineering·~30m