🗺️ Data Engineering · Journey Playbooks

Playbook: Harden a streaming platform

Correct event-time semantics with bounded state and monitored watermarks.

journey~75 minData EngineersAnalytics EngineersPlatform Data Teams

Journey Steps

1Step 1 — Choose windowing and watermarks correctly: start with "Timestamp on event time extracted from payload, not arrival"
2Step 2 — Make batch pipelines rerunnable by design: start with "Key every write by logical date/partition; overwrite partitions, never append blind"
3How it fits together: Watermark lag tells the truth; dashboards lag it.

Commands

Install all referenced skills

$ npx skills add aniruddhaadak80/skills --skill streaming-modeling-stream-processing-windows && npx skills add aniruddhaadak80/skills --skill pipelines-batch-pipeline-idempotency

Tags

#playbook#journey#data-engineering

Install this skill

$ npx skills add aniruddhaadak80/skills --skill playbook-data-engineering-streaming-hardening -a opencode

SKILL.md

# Playbook: Harden a streaming platform

> Correct event-time semantics with bounded state and monitored watermarks.

## When to Use This Skill

Correct event-time semantics with bounded state and monitored watermarks.

## Steps

1. Step 1 — Choose windowing and watermarks correctly: start with "Timestamp on event time extracted from payload, not arrival"
2. Step 2 — Make batch pipelines rerunnable by design: start with "Key every write by logical date/partition; overwrite partitions, never append blind"
3. How it fits together: Watermark lag tells the truth; dashboards lag it.

View source on GitHub ↗

Take retrieval-augmented answers from empty repo to evaluated production feature.

🗺️ AI Engineering·~135m

Open →

🗺️Playbook: Harden an agent before giving it toolsjourney

Ship an autonomous agent whose failure modes are cheap, visible, and reversible.

🗺️ AI Engineering·~125m

Open →

🗺️Playbook: Cut LLM costs without quality collapsejourney

Reduce monthly inference spend measurably while keeping answer quality within tolerance.

🗺️ AI Engineering·~95m

Open →