Learn Java Kafka in Action

Kaufman-style skill map for mastering Java Kafka, Kafka Streams, ksqlDB, event-driven patterns, and production deployment without wasting effort on shallow template learning.

Kafka as Distributed Commit Log

19 min

Deep mental model of Kafka as a distributed append-only event log, including topics, partitions, offsets, ordering, retention, replay, fan-out, and why Kafka is not a normal queue.

Topic, Partition, Replica, Leader, ISR

Deep dive into Kafka topic, partition, replica, leader, ISR, durability, and topic design as production engineering primitives.

KRaft and Kafka Control Plane

Deep dive into Kafka KRaft mode, metadata quorum, controller roles, broker roles, control plane failure modes, and production operating model.

Java Client Architecture

Deep dive into Java Kafka client architecture: producer, consumer, admin client, networking, metadata, threading, serializers, interceptors, lifecycle, and production-safe usage patterns.

Producer Reliability and Delivery Semantics

Production-grade guide to Kafka producer reliability and delivery semantics: acks, retries, idempotence, ordering, timeouts, transactions, fencing, and failure reasoning.

Producer Throughput, Batching, and Compression

Producer throughput, batching, compression, memory pressure, latency trade-offs, and Java-side tuning model for production Kafka systems.

Partitioning and Key Design

Partitioning and key design for Java Kafka systems, covering ordering boundaries, entity affinity, skew, hot partitions, repartitioning, tenant strategy, and custom partitioner trade-offs.

Consumer Groups and Offset Ownership

Consumer group, partition ownership, offset commit, rebalance, and Java consumer lifecycle model for production Kafka systems.

Consumer Correctness Patterns

Consumer correctness patterns for at-most-once, at-least-once, effectively-once, idempotency, deduplication, ordering, and safe side effects.

Error Handling, Retry Topics, and DLQ

Error handling, retry topics, dead letter queues, poison pill isolation, offset discipline, replay operations, and failure policy for production Java Kafka consumers.

Schema Contracts and Evolution

Schema contracts, Schema Registry, Avro, Protobuf, JSON Schema, compatibility modes, subject naming, governance, and safe event evolution for Java Kafka systems.

Event Design and Versioning

Event design and versioning for Java Kafka systems: event envelope, business facts, causality, timestamps, identifiers, semantic evolution, regulatory-grade auditability, and safe topic/event lifecycle design.

Communication Patterns with Kafka

Communication patterns with Kafka for Java systems: pub/sub, event-carried state transfer, commands, request/reply, choreography, brokered integration, coupling analysis, and architecture decision rules.

Data Patterns: CDC, Outbox, Event Sourcing

24 min

Data patterns for Kafka-based Java systems: CDC, transactional outbox, event sourcing, CQRS, compacted topics, replay, snapshots, auditability, and consistency boundaries.

Pipeline Topology Design

23 min

Pipeline topology design for Java Kafka systems: source-transform-sink architecture, fan-out/fan-in, branching, enrichment, ownership, contracts, lineage, replay, scaling, and operability.

Kafka Streams Core Model

23 min

Kafka Streams core model for Java engineers: topology, KStream, KTable, GlobalKTable, tasks, stream threads, state stores, changelog topics, repartition topics, restoration, scaling, and failure reasoning.

Streams DSL vs Processor API

Kafka Streams DSL versus Processor API for advanced Java engineers: abstraction choice, topology control, custom processors, state stores, punctuators, testing, naming, error boundaries, and production design trade-offs.

Windowing and Time Semantics

24 min

Windowing and time semantics in Kafka Streams for advanced Java engineers: event time, processing time, stream time, timestamp extractors, tumbling, hopping, sliding, session windows, grace period, late events, suppression, retention, testing, observability, and production review discipline.

Stateful Processing, Joins, and Aggregations

27 min

Stateful processing, joins, and aggregations in Kafka Streams for advanced Java engineers: KGroupedStream, KTable, reduce, aggregate, count, stream-table duality, repartition topics, changelog topics, state stores, joins, foreign-key thinking, tombstones, compaction, testing, observability, and architecture review discipline.

Exactly-Once Stream Processing

19 min

Exactly-once stream processing in Kafka Streams for advanced Java engineers: processing guarantees, transactions, idempotent production, state-store consistency, changelog coordination, output correctness, external side-effect boundaries, failure matrix, testing, observability, and architecture review discipline.

ksqlDB Mental Model

17 min

ksqlDB mental model for advanced Java Kafka engineers: SQL over Kafka, STREAM vs TABLE, persistent queries, push queries, pull queries, materialized views, Kafka Streams relationship, deployment architecture, query lifecycle, schema/key semantics, operational boundaries, and decision framework.

ksqlDB Stream/Table Query Design

22 min

ksqlDB stream/table query design for advanced Java Kafka engineers: CREATE STREAM/TABLE, CSAS/CTAS, persistent queries, repartitioning, joins, windowed aggregation, key formats, schema inference, materialization, push/pull query access, lifecycle management, testing, performance, and production governance.

Kafka Connect Integration Architecture

22 min

Kafka Connect integration architecture for advanced Java Kafka engineers: source and sink connectors, workers, tasks, distributed mode, converters, Schema Registry, SMTs, offset/config/status topics, connector lifecycle, DLQ/error handling, backpressure, observability, security, deployment, and production runbooks.

Batching, Backpressure, and Flow Control

19 min

Batching, backpressure, and flow control for production-grade Java Kafka systems: producer batching, consumer polling, fetch tuning, downstream pressure, bounded concurrency, adaptive throttling, retry pressure, and operational control loops.

Ordering, Consistency, and Idempotency

17 min

Ordering, consistency, and idempotency for production-grade Java Kafka systems: partition ordering, causal ordering, per-aggregate sequencing, duplicate suppression, idempotent side effects, consistency boundaries, projection correctness, and regulatory-grade workflow invariants.

Multi-Service Transaction Boundaries

Multi-service transaction boundaries for Java Kafka systems: local database transactions, Kafka transactions, transactional outbox, CDC, inbox pattern, saga, orchestration, compensation, auditability, and regulatory-grade failure modeling.

Security, ACL, SASL, mTLS, and Governance

Security, ACL, SASL, mTLS, and governance for production Java Kafka systems: threat modeling, encryption, authentication, authorization, service identity, tenant isolation, secrets rotation, topic governance, data classification, auditability, and security operations.

Observability, Metrics, Logs, and Traces

Observability handbook for production Kafka systems: metrics, logs, traces, lag, SLOs, alerting, dashboards, and incident runbooks.

Performance Benchmarking and Capacity Planning

17 min

Performance benchmarking and capacity planning handbook for Kafka platforms: throughput, latency, partitions, broker sizing, producer/consumer tuning, storage, and load test methodology.

Production Deployment Models

Production deployment models for Kafka across bare metal, virtual machines, containers, and managed cloud; includes topology, storage, networking, rack awareness, failure domains, deployment checklists, and architecture review templates.

Kafka on Kubernetes with Strimzi and Confluent Operator