Название: Fundamentals of Data Observability: Implement Trustworthy End-to-End Data Solutions (Final)
Автор: Andy Petrella
Издательство: O’Reilly Media, Inc.
Год: 2023
Страниц: 264
Язык: английский
Формат: epub (true)
Размер: 10.1 MB

Quickly detect, troubleshoot, and prevent a wide range of data issues through data observability, a set of best practices that enables data teams to gain greater visibility of data and its usage. If you're a data engineer, data architect, or Machine Learning engineer who depends on the quality of your data, this book shows you how to focus on the practical aspects of introducing data observability in your everyday work.

Author Andy Petrella helps you build the right habits to identify and solve data issues, such as data drifts and poor quality, so you can stop their propagation in data applications, pipelines, and analytics. You'll learn ways to introduce data observability, including setting up a framework for generating and collecting all the information you need.

As we venture into an era characterized by unprecedented data growth, the importance of understanding our data—its sources, destinations, usages, and behaviors—has never been more important. Observability, traditionally a term associated with software and systems engineering, has now made its way into the data space, becoming a cornerstone of trustworthy, efficient, and insightful data systems. This book aims to guide readers into the depth of this new and necessary discipline, exploring its principles, techniques, and evolving best practices.

Fundamentals of Data Observability is not just for data engineers or data scientists, but for anyone who interacts with data systems in their daily work life. Whether you’re a chief data officer (CDO), a chief technology officer (CTO), a manager, a leader, a developer, a data analyst, or a business manager, understanding data observability concepts and principles will empower you to make better decisions, build more robust systems, and gain greater insight from your data resources.

This book begins by outlining the core concepts of data observability, drawing parallels to similar concepts in software engineering, and setting the stage for the more advanced material. It subsequently delves into the principles and techniques to achieve data observability, providing practical guidance on how to implement them. The final section discusses how to get started today with the system you are using or have inherited. The book concludes with thoughts about the future of data observability, exploring ongoing research and emerging trends that are set to shape the field in the coming years.

Every chapter in this book is packed with actionable advice to reinforce the topics covered. My aim is not merely to impart knowledge but to facilitate the practical application of data observability concepts in your real-world situations.

I hope that by the end of this book, you not only will understand the “what” and “why” of data observability but will also be armed with the “how”—practical knowledge that you can apply to improve the reliability, usability, and understandability of your data systems.

• Learn the core principles and benefits of data observability
• Use data observability to detect, troubleshoot, and prevent data issues
• Follow the book's recipes to implement observability in your data projects
• Use data observability to create a trustworthy communication framework with data consumers
• Learn how to educate your peers about the benefits of data observability