Prevent data leakage in ML pipelines

Steps

1. 1Split temporally when production predicts the future
2. 2Fit scalers/imputers/encoders inside folds only, never full data
3. 3Ban features unavailable at prediction timestamp (audit with data team)
4. 4Group correlated rows (same user/device) into same split side
5. 5Track feature computation code versions with model artifacts
6. 6Validate: offline-vs-online correlation checked each release

Common Pitfalls

• ▲Random shuffles leaking user histories across train/test
• ▲Target encoding fit on full dataset

Commands

$ npx skills add aniruddhaadak80/skills --skill ml-modeling-leakage-prevention

$ npx skills add aniruddhaadak80/skills --skill ml-modeling-leakage-prevention -g

Tags