Название: Stream Processing: Hands-on with Apache Flink
Автор: Giannis Polyzos
Издательство: Leanpub
Год: 2023-08-10
Страниц: 234
Язык: английский
Формат: pdf (true), epub
Размер: 28.8 MB

Get onboard this journey into the land of streams. This is a complete hands-on book about Apache Flink, that follows real-life use cases and will help you learn how to create scalable end-to-end stream processing pipelines.

This is a complete hands-on book about Apache Flink.

The book follows real-life use cases and you will learn how to create end-to-end stream processing pipelines.

We will be using Redpanda and Apache Kafka - along with other technologies - so an understanding of Apache Kafka and Redpanda concepts like topics/partitions and producers/consumers is nice to have.

Apache Flink is the defacto solution when it comes to low-latency stream processing, but its unified API allows also for historic data processing with no code changes.
Moreover, Flink is proven in production at an extremely large scale and it also has a rich ecosystem with a bright future ahead.

The book is designed to teach you the theory and the practicals as fast as possible. The reader should be able to get from zero to production-ready applications fast with enough practice on the concepts introduced in the book, along with having enough knowledge to debug and troubleshoot when things go wrong.

Typically, streaming architectures can include:
• Streaming Layers like Apache Kafka, Redpanda and Apache Pulsar
• Stream Processing Engines like Apache Flink
• Realtime Analytical Data Stores like Apache Pinot, Clickhouse and StarRocks
• Datalake Table Formats like Apache Paimon and Apache Hudi, Apache Iceberg and Delta Lake
… and more.

In real-life production systems, you typically consume the data from a streaming layer like Apache Kafka, Redpanda, or Apache Pulsar. Apache Kafka is the most popular and the industry standard. Being born in the modern data era both Redpanda and Apache Pulsar come with a few benefits over Apache Kafka, but a comparison between these systems is beyond the scope of this book. In this book, we will be using Redpanda since it’s Kafka-compatible. So why Redpanda and not Kafka directly? Personally, I enjoy using Redpanda and I also like working with the Redpanda Web console. There are also many people currently running Flink on Kubernetes and both Redpanda and Apache Pulsar are built for such environments.

Hope you will enjoy it and use it as a guide in your journey in the land of streams.