Choose windowing and watermarks correctly

Steps

1. 1Timestamp on event time extracted from payload, not arrival
2. 2Set watermark delay from measured lateness distribution p99, not guesswork
3. 3Decide allowed-lateness policy: update vs side-output for stragglers
4. 4Key by cardinality you can hold state for; watch idle-key eviction
5. 5Verify exactly-once end-to-end through failure injection tests
6. 6Monitor watermark lag as the primary stream health metric

Common Pitfalls

• ▲Processing-time windows rewriting history on replays
• ▲Unbounded state growth from high-cardinality keys

Commands

$ npx skills add aniruddhaadak80/skills --skill streaming-modeling-stream-processing-windows

$ npx skills add aniruddhaadak80/skills --skill streaming-modeling-stream-processing-windows -g

Tags