Kafka, The definitive Guide

Livro digital

Título:
Kafka, The definitive Guide

Autor:
Neha Narkhede

Categoria:
Tecnologia > Cloud

Doador:
Raffaello D. N.

Sinopse:
Distributed systems that process real-time data streams at scale require more than just deploying a message broker — they demand understanding how producers commit messages reliably, how consumers track offsets through partition rebalances, and how the cluster itself coordinates replication under failure. This guide, written by the engineers who helped build Kafka at LinkedIn and Confluent, builds that understanding from the ground up, starting with broker hardware selection and OS tuning before moving into the internals of request processing, physical storage layout, log compaction, and controller election. The book dedicates equal depth to both sides of the wire: producers with custom serializers, Avro integration, and partition strategies; consumers with the poll loop, offset commit semantics, and rebalance listeners. Two standout sections cover reliable data delivery — replication factors, unclean leader election, and exactly-once semantics — and building production data pipelines with Kafka Connect, including real-world connector examples for file sources, MySQL-to-Elasticsearch migration, and stream processing alternatives. With chapters on cross-cluster mirroring architectures, administering topics and consumer groups, and monitoring broker metrics in production, this guide serves both the operations engineer tuning a live cluster and the application developer designing a data pipeline. The operational insight the authors bring — earned through years of running Kafka at scale — is what separates a working deployment from a reliable one.