⚡AgentSkills
🧮 ML Research Engineering · Inference Optimization

Tune speculative decoding without regressing quality

Drafter selection, acceptance-rate monitoring, and batch-interaction effects in production serving.

advanced~45 minResearch EngineersML ScientistsPhD Researchers

Steps

  1. 1Match drafter to target on YOUR distribution; measure acceptance rate per task class
  2. 2Tune speculation length by observed acceptance — longer drafts waste on rejection
  3. 3Watch batching interaction: gains shrink as batch size fills GPUs
  4. 4Verify output distribution equivalence against target-only sampling
  5. 5Re-evaluate whenever either model updates; pairs drift together
  6. 6Track tokens-per-second AND cost-per-token separately; they optimize differently

Common Pitfalls

  • ▲Acceptance rates from benchmarks failing on domain jargon
  • ▲Quality regressions hidden inside 'equivalent' temperature settings

Commands

Install with skills CLI
$ npx skills add aniruddhaadak80/skills --skill inference-optimization-speculative-decoding-tuning
Install globally
$ npx skills add aniruddhaadak80/skills --skill inference-optimization-speculative-decoding-tuning -g

Tags

#speculative-decoding#inference#llm#ml-research#inference-optimization

Related skills

Read scaling-law claims like a practitioneradvanced

Compute budgets, data-wall caveats, and extrapolation ranges separating signal from slide-ware.

🧮 ML Research Engineering·~35m
Open →