Notes on software engineering, distributed systems, and using AI to build real things.https://zidan.me/https://zidan.me/blog/idempotency-series/part-3-idempotent-consumers-and-pipelines/https://zidan.me/blog/idempotency-series/part-3-idempotent-consumers-and-pipelines/The idempotent consumer pattern implemented (processed-message table in the same transaction), dedup identity for events, Kafka's exactly-once machinery taken apart honestly, offset-commit ordering, and the series-capstone retrofit checklist.Thu, 18 Jun 2026 00:00:00 GMThttps://zidan.me/blog/idempotency-series/part-2-building-the-key-store/https://zidan.me/blog/idempotency-series/part-2-building-the-key-store/Brandur Leach's Stripe-style Postgres key store walked end to end in Kotlin: the schema, atomic claim via ON CONFLICT, recovery points so a crashed request resumes instead of double-executing, the Redis variant compared honestly, and the window you can't close.Mon, 15 Jun 2026 00:00:00 GMThttps://zidan.me/blog/idempotency-series/part-1-the-exactly-once-lie/https://zidan.me/blog/idempotency-series/part-1-the-exactly-once-lie/Why exactly-once delivery is impossible, the at-most-once vs at-least-once dichotomy, idempotency keys (who mints them, where they live, when they expire), and a taxonomy of operations by natural idempotency.Fri, 12 Jun 2026 00:00:00 GMThttps://zidan.me/blog/distributed-clocks-series/part-3-hybrid-clocks-in-production/https://zidan.me/blog/distributed-clocks-series/part-3-hybrid-clocks-in-production/Hybrid logical clocks walked by hand, CockroachDB's uncertainty intervals and read restarts, Spanner's TrueTime and commit-wait, Cassandra's last-write-wins as a cautionary tale, and a decision sketch for choosing a clock.Sun, 07 Jun 2026 00:00:00 GMThttps://zidan.me/blog/distributed-clocks-series/part-2-logical-clocks/https://zidan.me/blog/distributed-clocks-series/part-2-logical-clocks/Lamport clocks, vector clocks, the version-vector distinction almost every article gets wrong, the sibling-explosion failure that motivated dotted version vectors, and interval tree clocks for dynamic membership.Sun, 10 May 2026 00:00:00 GMThttps://zidan.me/blog/distributed-clocks-series/part-1-wall-clocks-and-causality/https://zidan.me/blog/distributed-clocks-series/part-1-wall-clocks-and-causality/Why physical timestamps can't order distributed events — NTP slewing and stepping, leap-second carnage, monotonic vs wall clocks, and Lamport's happened-before as the only order a distributed system can compute without coordination.Sun, 12 Apr 2026 00:00:00 GMThttps://zidan.me/blog/crdts-from-scratch/part-3-use-cases-and-libraries/https://zidan.me/blog/crdts-from-scratch/part-3-use-cases-and-libraries/Where CRDTs ship today (Yjs, Automerge, Loro, Riak, Redis), the metadata and invariant costs, and when not to reach for one.Tue, 03 Feb 2026 00:00:00 GMThttps://zidan.me/blog/crdts-from-scratch/part-2-the-crdt-zoo/https://zidan.me/blog/crdts-from-scratch/part-2-the-crdt-zoo/Building the classic CRDTs in Go: G-Counter, PN-Counter, LWW- and MV-Registers, G-Set / 2P-Set / OR-Set, and an RGA sketch for collaborative sequences.Mon, 26 Jan 2026 00:00:00 GMThttps://zidan.me/blog/crdts-from-scratch/part-1-foundations/https://zidan.me/blog/crdts-from-scratch/part-1-foundations/The intuition and the math behind Conflict-free Replicated Data Types: Strong Eventual Consistency, join semilattices, and the difference between state-based and operation-based replication.Sun, 18 Jan 2026 00:00:00 GMThttps://zidan.me/blog/event-sourcing-from-the-ground-up/part-4-production-realities/https://zidan.me/blog/event-sourcing-from-the-ground-up/part-4-production-realities/Schema evolution and upcasting, the right to be forgotten, why Kafka isn't an event store, dynamic consistency boundaries, and the 2026 JVM library landscape.Sun, 16 Nov 2025 00:00:00 GMThttps://zidan.me/blog/event-sourcing-from-the-ground-up/part-3-cqrs-and-projections/https://zidan.me/blog/event-sourcing-from-the-ground-up/part-3-cqrs-and-projections/Splitting reads from writes: projections as folds over the log, checkpoints and rebuilds, living with eventual consistency, and integration events.Mon, 10 Nov 2025 00:00:00 GMThttps://zidan.me/blog/event-sourcing-from-the-ground-up/part-2-building-it-in-kotlin/https://zidan.me/blog/event-sourcing-from-the-ground-up/part-2-building-it-in-kotlin/A from-scratch Kotlin event store with optimistic concurrency, a bank-account decider, replay, and snapshots — about 150 lines, no framework.Fri, 07 Nov 2025 00:00:00 GMThttps://zidan.me/blog/event-sourcing-from-the-ground-up/part-1-foundations/https://zidan.me/blog/event-sourcing-from-the-ground-up/part-1-foundations/The intuition behind event sourcing: events vs commands vs state, the event store contract, the benefits and the costs that compound, and a precise comparison with CRDTs.Sun, 02 Nov 2025 00:00:00 GMThttps://zidan.me/blog/turn-your-coding-agent-into-a-grounded-coding-assistant/https://zidan.me/blog/turn-your-coding-agent-into-a-grounded-coding-assistant/Using Docs-MCP-Server to ground a coding agent in real, up-to-date library docs so it stops hallucinating APIs that don't exist.Mon, 22 Sep 2025 00:00:00 GMThttps://zidan.me/blog/a-self-hosted-read-listen-ask-reader-my-journey-how-to-use-it/https://zidan.me/blog/a-self-hosted-read-listen-ask-reader-my-journey-how-to-use-it/Stitching together a local EPUB reader, Orpheus TTS, and an LLM into a self-hosted reading workflow that fits how I actually read.Sun, 10 Aug 2025 00:00:00 GMThttps://zidan.me/blog/software-engineering-in-the-time-of-ai/https://zidan.me/blog/software-engineering-in-the-time-of-ai/AI is already part of the developer’s toolkit, but it’s not the revolution many are claiming. Its value lies in automation, not intelligence.Sun, 08 Jun 2025 00:00:00 GMThttps://zidan.me/blog/airflow-on-kubernetes-series/part-4-xcom-secrets-slack/https://zidan.me/blog/airflow-on-kubernetes-series/part-4-xcom-secrets-slack/Moving XCom payloads to object storage instead of bloating the metadata DB, managing secrets and connections the isolated-worker way, and wiring up Slack notifications for failing DAGs.Wed, 30 Apr 2025 00:00:00 GMThttps://zidan.me/blog/airflow-on-kubernetes-series/part-3-executors-and-lifecycle/https://zidan.me/blog/airflow-on-kubernetes-series/part-3-executors-and-lifecycle/When to use Local vs Celery vs Kubernetes (and multi-executor), the full task lifecycle, how a worker actually gets its task, and what happens when code changes mid-run.Fri, 25 Apr 2025 00:00:00 GMThttps://zidan.me/blog/airflow-on-kubernetes-series/part-2-database-pgbouncer/https://zidan.me/blog/airflow-on-kubernetes-series/part-2-database-pgbouncer/Why Airflow opens so many connections, how to size the SQLAlchemy pool, and why PgBouncer in transaction mode is non-negotiable for Postgres.Sun, 20 Apr 2025 00:00:00 GMThttps://zidan.me/blog/airflow-on-kubernetes-series/part-1-setup-helm-gitsync/https://zidan.me/blog/airflow-on-kubernetes-series/part-1-setup-helm-gitsync/The moving pieces of Airflow 3, the git-sync sidecar pattern for DAG delivery, and the image-rebuild-on-dependency-change workflow with Helmsman.Tue, 15 Apr 2025 00:00:00 GMThttps://zidan.me/blog/building-a-vision-language-api-to-convert-pdfs-into-markdown-with-smoldocling/https://zidan.me/blog/building-a-vision-language-api-to-convert-pdfs-into-markdown-with-smoldocling/Building a blazing-simple API on top of SmolDocling, an ultra-compact vision-language model that turns messy PDFs into clean, structured Markdown: tables, equations, code blocks, and all.Sat, 08 Mar 2025 00:00:00 GMThttps://zidan.me/blog/monitoring-next-js-with-prometheus-in-kubernetes/https://zidan.me/blog/monitoring-next-js-with-prometheus-in-kubernetes/Wiring Prometheus into a Next.js app via Server Actions and deploying the whole thing inside a Kind cluster.Mon, 20 Jan 2025 00:00:00 GMT