Название: Learning Apache Drill: Query and Analyze Distributed Data Sources with SQL
Автор: Charles Givre, Paul Rogers
Издательство: O'Reilly Media
ISBN: 1492032794
Год: 2018
Страниц: 331
Язык: английский
Формат: TruePDF +code
Размер: 10.16 MB

Get up to speed with Apache Drill, an extensible distributed SQL query engine that reads massive datasets in many popular file formats such as Parquet, JSON, and CSV. Drill reads data in HDFS or in cloud-native storage such as S3 and works with Hive metastores along with distributed databases such as HBase, MongoDB, and relational databases. Drill works everywhere: on your laptop or in your largest cluster.

At its core, Apache Drill is a SQL engine for big data. In practical terms, what this means is that Drill acts as an intermediary that allows you to query self­describing data using standard ANSI SQL. To use a comparison from the science fiction series Star Trek, Drill acts as a universal translator for your data and enables you to use SQL to interact with your data as if it were a table in a database, whether it is or not. Bringing this down to earth, Drill enables an analyst, armed only with a knowledge of SQL or a business intelligence (BI) tool such as Tableau, to analyze and query their data without having to transform the data or move it to a centralized
data store.

In this practical book, Drill committers Charles Givre and Paul Rogers show analysts and data scientists how to query and analyze raw data using this powerful tool. Data scientists today spend about 80% of their time just gathering and cleaning data. With this book, you’ll learn how Drill helps you analyze data more effectively to drive down time to insight.

- Use Drill to clean, prepare, and summarize delimited data for further analysis
- Query file types including logfiles, Parquet, JSON, and other complex formats
- Query Hadoop, relational databases, MongoDB, and Kafka with standard SQL
- Connect to Drill programmatically using a variety of languages
- Use Drill even with challenging or ambiguous file formats
- Perform sophisticated analysis by extending Drill’s functionality with user-defined functions
- Facilitate data analysis for network security, image metadata, and machine learning